Tensorflow 1.5 built with AVX support

TL;DR – download tensorflow 1.5 with AVX support from the link on the bottom of this post

When running machine learning code on a new hardware using libraries available on PIP we are not using all capabilities provided by our cpu:
2018-01-10 09:35:05.048387: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2

Last night I’ve rebuilt the tensorflow to support AVX CPU instructions. The set up for build takes about an hour. The build itself took 2 hours 20 minutes on my i7-8700k desktop with Windows 10 and hit the computer quite hard.

I’ve used official build manual (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/cmake/README.md), but it doesn’t mention all requirements:
* you need to install numpy in the environment you use for build
* you need to install wheel in the environment you use for build (otherwise it fails after 2 hours of build – sweet)
* if building against cuda9.1 you need to copy math_functions.h from cuda91/include/crt/ to cuda91/include directory (otherwise it fails after 1h of build)

The results?
Sample program without AVX:
start: 2018-01-10 09:35:04.609053
finish:2018-01-10 09:36:00.339329

total: ~55.5s

The same code with AVX:
start: 2018-01-10 09:36:18.167291
finish:2018-01-10 09:36:55.693329

total: ~37.5s

Here is the wheel file with support for AVX tensorflow_gpu-1.5.0rc0-cp36-cp36m-win_amd64.whl if you don’t want to run the build process itself.

And CPU usage during build (I got a new computer yesterday and I’m still excited by new toy :))


Adding wildcards to Google AIY actions on Raspberry Pi.

I’ve been playing with Google AIY on raspberry pi for nearly an hour now and I love it. If you are lucky you can get your kit from issue 57 of The MagPi Magazine.

Google provided a python based example app that recognises the command you spoke to the box and runs your action. The problem with it is that the command needs to match literally without and option to add variable part (a parameter). In real world I want to give parameters to the commands, for example “Add note my note“. So I’ve hacked the app to do just that. Here are the steps:
1. Modify the actionbase.py to recognize patterns. In class class KeywordHandler(object): change the handle method:

class KeywordHandler(object):

    """Perform the action when the given keyword is in the command."""

    def __init__(self, keyword, action):
        self.keyword = keyword.lower()
        self.action = action

    def get_phrases(self):
        return [self.keyword]

    def handle(self, command):
        if("*" in self.keyword):
            match = re.match(self.keyword.lower(), command.lower())
            if match:
                return True
            if self.keyword in command.lower():
                return True
                return False

2. Make sure the action you are running understands that the param given to it is the variable part. I’ve modified the SpeakAction to do just that:

class SpeakAction(object):

    """Says the given text via TTS."""

    def __init__(self, say, words):
        self.say = say
        self.words = words

    def run(self, voice_command):

3. Add new action in make_actor method:

    actor.add_keyword(_('add note (.*)'),SpeakAction(say,"adding $1"))

Have fun!


How to integrate Prestashop e-commerce site with Shopzilla

So you decided to integrate your shop with Shopzilla. If you are reading this in March 2014 then come back in a month to learn is it worth it. If you are decided then read on.

There are some commercial solutions available that will connect your shop to Shopzilla. The cost roughly 100 GBP. It’s not a huge sum, but I would advise you to spend the money on the Shopzilla ads instead to see if it is worth it. Which causes a small problem of uploading the products to Shopzilla.

Shopzilla’s interface seems to be stuck between 1999 and 2005. The only reasonable way they offer is to upload a CSV file for which they provide a documentation with some infuriating errors. And their system does not provide feedback on what you are doing wrong. Are you ready?

In order to create csv file they require I used SQL script and run it through PHPMySQL to save the result in csv format (please note you need to change it to .txt to upload to Shopzilla). Please save it as tab (\t) separated file without quotes around columns and with new line removal.

The script is below. You will have to adjust:
* case section to provide your category mapping
* change yourshop to base url of your shopin product and image url
* adjust id_lang in where statement to change the language of the export. I’m using 1 for English
* quantity in stock is hardcoded to 1000. That will be fine in most cases.
* Leave the shipping cost and bid empty, because Shopzilla suggests setting those parameters by the admin panel, not the feed itself.

The resulting txt file works fine with SEO optimized urls.

when cat.name like 'Earrings' then '14138'
when cat.name like 'Bracelets' then '14135'
else '14157'
end as Category,
m.name AS Manufacturer,
pl.name AS Title,
') AS 'Product Description',
concat('http://yourshop/',cat.link_rewrite,'/',cast(p.id_product as char),'-',pl.link_rewrite,'.html') as Link,
concat('http://yourshop/',cast(img.id_image as char),'-thickbox_default/',cast(img.id_image as char),'.jpg') as Image,
concat(m.name,'_',cast(p.id_product as char)) as SKU,
'1000' as Stock,
'New' AS 'Condition',
p.weight AS 'Shipping Weight',
'' as 'Shipping Cost',
'' as Bid,
'9 14' as 'Promotional Description',
'' as 'EAN / UPC',
p.price as Price

FROM ps_product p INNER JOIN
ps_product_lang pl ON p.id_product = pl.id_product LEFT JOIN
ps_manufacturer m ON p.id_manufacturer = m.id_manufacturer LEFT JOIN
ps_image img on p.id_product = img.id_product LEFT JOIN
ps_category_lang cat on p.id_category_default = cat.id_category
where img.position=1
and pl.id_lang=1
and cat.id_lang=1
and p.active=1

Please let me know what your results in using Shopzilla are. Was it worth for you?


Mapping sales in UK using google geochart

My Jewellery shop generates slow but steady flow of orders, which lets me play with data. Today I’ve try mapping the shipping addresses to a map.

I’ve decided to try Google’s geochart from charts API. There’s simply no way they could make it any easier. They have even created a playground where you can test the api from the browser.

There are some issues with this chart though – the map looks bland, it can’t be zoomed and there’s not real map overlay. Also it take ages to draw unless the markers are given in lat/lon format.

Here’s the result:
orders in uk map


Raspberry Pi, servo motor, gpio, i2c and soldering weekend

This weekend I decided to work on my soldering skills and finally assemble the PWM driver PCA9685 I bought from Adafruit couple weeks back. Not all solder points looks perfect, but I’ve manage not to burn the PCB which I consider a major success. I thank all the
guys how put soldering tutorials on the youtube!


Then I played with software.
* I2C: I used python to steer servo controller via I2C interface. For some reason the Raspbian image has I2C kernel module disabled, so I had to comment blacklist i2c-bcm2708 in /etc/modprobe.d/raspi-blacklist.conf and the add i2c-dev and i2c-bcm2708 to /etc/modprobe to enable them to start.
* GPIO: Raspbian has all libraries loaded by default for GPIO development in python. I had encountered hardware problem instead – there’re many sources on the web describing pin layout of GPIO port, but non of them says which pin is the physical pin 1! I have some gaps in basic knowledge, so I have missed a small rectangle marking P1 – it’s the one nearest to the side of the board in bottom row. See a picture with pins P3, P5, P9, P10 and P11 connected:

And here it is – Raspberry Pi waving The Flag of the United States of America


MGPlayer on Raspberry PI (javafx!)

I’ve just “successfully” run my MG Podcast Player on Raspberry Pi using just released JDK8 for ARM Preview.

Performance isn’t great compared to desktop system and it doesn’t actually play mp3 files (Media are not supported yet), but having ability to run a java 8 application on $35 is amazing by itself!

If you want to try follow the steps on oracle site to install the image and then run:
$ /opt/jdk1.8.0/bin/java -Djavafx.platform=eglfb -jar MGPlayer.jar

* -Djavafx.platform is crucial as it let javafx work on OpenGL ES 2.0 embedded device
* I’ve installed java on my raspbmc image