Cows and Runes

We tried a new area to hike recently and were surprised at this gem in the Stockholm suburbs. The area is called Järvafältets naturreservat and it is near the Akalla T-bana line. The area is quite large and its pretty easy to go for a more than 20km walk. Its made up of a mix of farmland, small lakes, woodland and small hills.

There are a lot of paths so easy to get a bit disorientated, but they do have a great map that you can pick up at entrances. The paths are not as well marked as the more isolated areas like the Sörmlandsleden, but you are never far from the city. It’s always a good idea to take some navigation device so you can orientate yourself even when close to the city  (eg. smartphone with maps and data connection, compass, GPS).

Jarvafaltet Map
Jarvafaltet Map

At the entrance near Akalla we saw this rune… there are many more in the park but we were pretty unsuccessful in finding them, even with the map and a lot of hunting. We assumed that mostly they are not as spectacular as this one and a bit hidden off the path.

Rune
Rune

jarvafalet rune map

And here is a spectacular cow we saw in a field.

Cow in Jarvafaltet
Cow in Jarvafaltet

and some of the landscape.

trees in jarvafaltet
trees in jarvafaltet
sabysjon
sabysjon

Photos of tiny things

When doing projects photographing them is part of the enjoyment. And for most of my hobbies things are very small so a macro lens is a requirement for a good photo. I don’t have one for my SLR, and for a quick snap I often just grab my iPhone. But of course its ability to to take a close up shot is limited. So I was very happy to find this set of clip on lenses for smartphones as a hobby show this weekend.

lenses1

For $12 I managed to get a fisheye, macro and wide angle lens in a set.

Here’s a first shot with the macro. Its the head of a map pin, laid on a sheet of printer paper, taken with an iPhone6.

pin1

Next I have to work on lighting but hopefully this will improve the pictures that I post.

Frying Pan Savers

We saw this concept in a high end cooking store and just had to duplicate them for a fraction of the price.

Get some cheap felt (real or artificial) from your local craft store… I purchased this hideous colour from the sale section.. it will be in the drawer not on display. Using a plate or pot draw a circle representing the inside of the pan, then draw a larger circle that represents the outer of the pan plus about 1 1/2 to 2  inches. Cut out around the outer circle and then make 8 wedge shaped cuts to the inner circle. Don’t worry too much about how accurate you are.

pansaver1

Make a set that matches your frying pan collection and now you can stack them without scratching the Teflon.

pansaver2

Total cost was about $5 for 3

Simple Roast Vegetable Mix

If you are like me you love roast vegetables but you usually have them with a joint of meat or a fowl. What if you want them more often that. Well they are actually super quick and easy.

Heat your oven to 450F (230C) and while it is warming chop the vegetables coarsely… no need for finess as long as they are roughly the same size. A great mix is potato, sweet potato, carrot and parsnip. Only the sweet potato needs to be peeled as the rest are delicious with the skin on. Here they are being washed… then dry them. They will all cook in about the same time.

Roast Veg1

Put a very tiny amount of oil in the bottom of an oven proof glass baking dish so that they don’t stick, and use a tiny bit more on your (clean) hands and mix them up a bit.  Add pepper and pop the dish in the oven at the top for about 15 minutes. if you prefer you can get away with no oil. Feel free to add other seasonings (we love to add curry powder or chilli for example)

Roast Veg 2

In the middle of cooking mix them up a bit more with a spatula and put back in for another 10 minutes.

Roast Veg3

Tasty!

Follow the Orange Paint path

It was a nice autumn day so we decided to go on a bit of a hike. We  have a wealth of choices here and we thought to try a bit more of the Sörmlandsleden (in english) which is a 1000km trail to the south of Stockholm. Its lovingly maintained by a group of volunteers who as part of that mark it with orange paint. The trail is generally pretty good for hiking with paths near the trailheads leading to more rugged territory. You can get to the trailheads on public transport and there are something like 100 sections. At various places along the trail there is access to fresh water and shelters. And you can exercise your Swedish ‘right to roam‘ and camp.

sormslandsleden2

For 200 SEK you can join and get access to the maps which are very good and a worthwhile investment for your own safety and to keep the volunteers supplied with paint and lumber.

As you can see, the trail is well marked and where necessary there are bridges across any watercourses that you need to traverse.

sormslandsleden1 sormslandsleden3

We have hiked other parts but today we decided to start at the beginning and hike stages 1 & 2, totaling about 15km and taking about 4 hours. It was great to get out and experience the fresh air and the countryside.

sormslandsleden4 sormslandleden5

To get to the trailhead we took the T-bana to Björkhagen and to return we caught the 573 bus from Nyfors. There are circular routes, although a lot are linear but they tend to go between public transport locations so you can leave the car at home.

Thanks to all those volunteers who maintain this fabulous asset that is so accessible.

Real Time Clock accuracy

I have a real time clock attached to my RPi and I thought that it would be an interesting project to monitor its accuracy. The clock I am using is a Maxim DS1307… this type does not have any fancy compensation, but from what I read it should be around 10ppm.

So I built a program for monitoring the clock and comparing it with a reference. For a reference I am using the RPi system clock slaved to NTP… which should provide synchronisation to within a few milliseconds. The linux clock of course reports to quite a fine resolution (1 microsecond ticks)… but the DS1307 only reads out to the 1 second level which means on the face of it that a fairly long run would be needed to make an estimate of the clock accuracy.

Here is a table of expected deviation in seconds for clocks of various accuracies… as you can see we would have to wait for 10 days to get a result from a 10ppm clock with a significant offset. The problem being after 1 or 2 days the error might be 1 second, and the resolution of the measurement is +/-1 second… not great for measuring drift.

clockerrortheory1

So I thought a bit more about the problem of measuring. We actually have more info than this as we can measure the time on the RPi to a resolution of 1 microsecond and we can measure frequently. So while we run we can compare the time from the real time clock to that more accurate clock. (it is accurate as it is set from ntp and it has finer resolution). Of course the rounding implied in the RTC result compared to the system clock will give  a a false sense of accuracy.  So what I decided to do was to round both and compare to only 1 second resolution. You might ask how does this help? Well what we can do now is see the statistics of the comparison at 1 second resolution and use them to see the drift rate.

First we set the RTC to be the same as the system clock set by ntp. Then we read both and compare the seconds. Initially we will get an answer of 0, with the occasional -1 or 1 due to the rounding. Gradually as the RTC drifts the number of reads that say 1 will increase (assuming it is running slower than the system time.This gives us an estimate of how far apart they are. In effect we are looking at the ‘density’ of rounding transitions up or down over time to estimate the drift.

This looks like the following;

The y axis is the delta and the x axis is the run time… here I picked 256 samples of a part of the run about 1000 readings into the process.

clockerrortheory2

So now time for a  test run. I started the program and let it log to a file. Unfortunately after about 2 days the RPi crashed … but after all that was the point of looking at this method… to avoid the need for very long runs. Here is the picture of the just over 11,000 points taken every 15 seconds.

clockerrortheory3

Now we can see a clear trend of drift between the clocks. There is a fair amount of noise due to the rounding jitter so we’ll need some filtering a to derive a trend. I applied an average of the offset (last 64, 128,  256 and 512 samples), calculated the sliding average  and plotted a trend in excel to see if this technique would work. Here is the result:

clockerrortheory4

It looks like we have about 8ppm drift from this result. of course we could try estimate to more accuracy, but that would require more detailed examination of the filter length and noisiness of the signal to come up with something reliable… I did a quick check of the results and 1ppm seemed to be about the right level of resolution as with the different filters the slope was varying by about 0.5ppm. While running this test  the temperature variation was about +/- 2C and the ntp was locked to a stratum 2 clock.

So in conclusion, in 2 days we managed to get an estimate of clock drift using the trend line of the offsets even though the absolute drift was only 2 seconds and the resolution of the RTC is 1 second. Next we need to test this for a longer period of time, and build in the filters and calculations to remove the manual processing steps in excel. Plus look more into the ntp stats for the local clock drift compensation as we are measuring versus the system clock slaved to ntp.

Here is the code for the program.


#!/usr/bin/env python
#
# ClockTest Program
# original and driver by John C. Shovic, SwitchDoc Labs
# 07/10/2014
# Modified by hobbyistdave, Oct 2015
#
# imports

import sys
#sys.path.append ('./lib')              # path for local libraries
import time
import datetime
import requests   # HTTP for humans
                  # by Kenneth Reitz  docs.python-requests.org
import SDL_DS1307 # the RTC driver

# setup the info needed to send the results to thingspeak

thingspeakurl = "https://api.thingspeak.com/update"   # url for sending updates to thingspeak
apikey = "1234567890123456"     # thingspeak api address for this channel 
ioTInterval = 15                # interval for maximum writes to thingspeak in seconds (min interval is 15 seconds)
url = thingspeakurl

def writeToIoT(payload):        # function to write to IoT
    try :
        r = requests.get(url, params=payload)  # send to thingspeak
        print time.time(), "  ", r.text, "  ", ntpSecs ,"   ", rtcSecs ,"   ", deltaSecs ,"      ", runTime , " ", r.status_code  
        #r.text will be the data point number
        #r.status code should be 200 if the page posted correctly
    except requests.exceptions as e :
            print e # some simple error trapping to avoid the program timing out
    except requests.exceptions.ConnectionError as e1 :
            print e1 # some simple error trapping if the connection is lost
    return

# Main Program

print ""
print "ClockTest, hobbyistdave"
print ""
print "Program Started at: "+ time.strftime("%Y-%m-%d %H:%M:%S")

filename = time.strftime("%Y-%m-%d%H:%M:%SRTCTest") + ".txt"
startTime = datetime.datetime.utcnow()
#print startTime
print "Writing system time to RTC to make them equal at start of test"
ds1307 = SDL_DS1307.SDL_DS1307(1, 0x68)
ds1307.write_now()


try :    # wrapper to allow clean exit
    # now setup the command shell for printing the results
    print " "
    print "---------------------------------------------------------------------------"
    print " "
    
# Main Loop - sleeps 1 seconds, then reads and prints values of all clocks
# checks time since last update and writes to IoT

    startTime = datetime.datetime.now()
    print "startTime : ", startTime
    print " "

    print "---------------------------------------------------------------------------"
    print "  Time          Point    NTP     RTC   Delta    Runtime         Response   "
    print "  (s)                   (s)     (s)   (s)                                  "
    print "---------------------------------------------------------------------------"

    ioTTimeStamp = time.time()
    lastDelta=0   # initialise delta
    while True:

        ntpTime = datetime.datetime.now()
        rtcTime = ds1307.read_datetime()
        #deltaTime = ntpTime - rtcTime
        ntpSecs = datetime.datetime.now().strftime('%S')
        rtcSecs = rtcTime.strftime('%S')
        #print ntpTime, "  ", rtcTime, "  ", deltaTime
        deltaSecs =  int(ntpSecs)  - int(rtcSecs)
        if lastDelta < 30 and deltaSecs <-30 : # fix instances where the seconds wrap across a minute boundary deltaSecs = deltaSecs+60 # we dont expect time to drift apart by more than 30 seconds elif lastDelta > -30 and deltaSecs >30 :
            deltaSecs = deltaSecs-60
        lastDelta = deltaSecs  # remeber the last delta

        runTime = ntpTime - startTime

        payload = {'key' : apikey, 'field1' : ntpTime , 'field2' : rtcTime , 'field3' : deltaSecs , 'field4' : runTime  } # make the payload
        if time.time() >= (ioTTimeStamp + ioTInterval):  #check if enough time has passed before updating thingspeak
            writeToIoT(payload)
            ioTTimeStamp = time.time()
        else:
            time.sleep(0.001)

        time.sleep(1)

except KeyboardInterrupt:
    print " "
    print "Exiting program due to Keyboard Interrupt"
    print " "
    time.sleep (1)
finally:
    print "----------- The End----------------"



Mirroring command line output to a file

Quite often when you are running a program, you would like to capture its output into a file for later analysis. I want to do this for a real time clock accuracy monitoring project that I am in process of running. Of course you can write to the file directly from your program, but there is a neat trick to both write to the standard command output, capture to a file and monitor the output at the same time.

First you run your program with the output redirected into a file;


sudo python -u myprog.py > proglog.txt

The > is the redirect and the python -u switch means do not buffer. Without this you will not see your log file change for a while as the OS will buffer a significant amount of the output.  When you write directly to the command line the buffer is flushed every line, but not when you redirect. In the Debian wheezy used on the RPi the buffer is 4kbytes so if you redirect to a file you wont see output for a while if you don’t use the -u switch. This is a common problem and makes for a confusing time as you see the log file size stays at 0 while the buffer fills.

Then in another terminal window use;


tail -f proglog.txt

You will now see the last 10 lines of the file that is being created with the output. As the file is updated the ‘tail’ will ‘follow’ it continuously so you get the benefit of writing to the console and of logging to a file. You can increase the number of lines tailed with the -n switch followed by the number of lines you require.

Another way you can do this is to ‘tee’ the output so that the file is generated and the command line shows in one terminal.

sudo python -u myprog.py  | tee proglog.txt

or

sudo python -u myprog.py 2>&1 | tee proglog.txt 

The second version with 2>&1 sends both the standard output and the standard error to the file.

These files are great for post processing the data, for example by importing them into excel. If you want to append lines to the last file rather than create the file new each time, use >> rather than >.


sudo python -u myprog.py >> proglog.txt

Spicy Swedish Pizza Salad

While on the Stockholm archipelago island of Utö we stopped at a small restaurant and ordered some pizza for lunch and with it we were offered pizza salad. After checking that we heard correctly as we had never heard of it we agreed and what showed up was a kind of vinegar coleslaw. Here’s my attempt at it.

First take about a quarter of a cabbage (500g) and chop it into sections that will go into the blender.

pizzasalad1

Be very careful here with the knife as cabbage can be quite tough to cut and its easy to slip.

pizzasalad2

Then chop them to the size that suits you.  Add vinegar to your taste (we used about 1/2 a cup of the strong vinegar Winborgs Attiksprit, diluted a bit as it is 12%). Then add fresh ground pepper to spice it up. Add a teaspoon of oil and a little salt (not too much as you want this to be crunchy, not break down into Sauerkraut).

pizzasalad4

To add a little colour chop up half of a red pepper and add it.

pizzasalad5

Leave in the fridge overnight to marinate. And now you have some spicy Swedish pizza salad to go next to your pizza. Its a great way to add some vegetables to a delicious meal of pizza and french fries (chips).