20180106

2017: Year in Review

I’m trying to make these posts a tradition (even if a few days late). I thought 2016 had been a really weird and fun year, but 2017 has beaten it easily. And I only hope 2018 will be even better in every way. For the record, when I say we, it means Laia and me unless explicitly changed.

Beware, some of the links are affiliate links. I only recommend what I have and like though, get at your own risk :)

Work

Everything work related has gone up. More work, better work, more interesting work. Good, isn’t it?

As far as my consulting job in London, the most relevant parts would be:

  • Lead a rewrite and refactor of the adserver (Golang) to improve speed and reliability.
  • Migrated a batch job from Apache Pig to Apache Spark to be able to cope with larger amounts of data from third parties (now we process 2x the data with 1/10th of the cost).
  • Planned an upgrade of our Kafka cluster from Kafka 0.8.2 to Kafka 0.10.1, which we could not execute as well as planned because the cluster went down. Helped save that day together with the director of engineering when that happened.
  • Was part of the hiring team, we’ve had one successful hire this year (passed probation, is an excellent team member and loves weird tech). Hopefully we enlarge our team much more in the coming year.
  • Put a real time service in Akka in production, serving and evaluating models generated by a Spark batch job.
We also moved offices, now we have a free barista “on premises”. Free, good quality coffee is the best that can be done to improve my productivity.

In April I got new business cards (designed by Laia, you can get your own design if you want, contact her):


I kept on helping a company with its SEO efforts, and as usual patience works. Search traffic has improved 30% year-to-year, so I’m pretty happy with it. Let’s see what the new year brings.

I became technical advisor of a local startup (an old friend, PhD in maths is a founder and works there as data scientist/engineer/whatever), trying to bring data insights to small and medium retailers. I help them with technology decisions where I have more hand-to-hand experience, or know where the industry is moving.

Life

Traveling up and down as usual (2-3 weeks in London, then l’Arboç, then maybe somewhere else…) sprinkled with some conferences and holidays.

Regarding life, the universe and everything, what I’ve done and where I’ve been
  • In February we visited Hay-on-Wye again, for my birthday
  • In March I convinced Holden Karau (was easy: she loves talking about Spark :D) to be one of our great keynote speakers at PyData Barcelona 2017
  • In late March we visited Edinburgh and Glasgow
  • In early May I attended PyData London to be able to prepare better for ours. Met some great people there.
  • A bit later in May I visited Lisbon for LX Scala, thanks Jorge and the rest for the great work
  • And at the end of May, we held PyData Barcelona 2017, where I was one of the organisers. We had more than 300 attendees, enjoying a lot of interesting talks. Thanks to all attendees and the rest of the organising committee... We made a hell of a great conference
  • Mid-June, I gave my first meetup presentation, Snakes and Ladders (about typing in Python as compared with Scala) in the PyBCN meetup
  • In late June, we visited Cheddar and Wells
  • In September I visited Penrith for the awesome (thanks Jon) Scala World 2017. Looking forward to the 2019 edition.
  • In early October we visited San Sebastian for the Python San Sebastian 2017 conference. We ate terribly well there (we can recommend Bodegón Alejandro as one of the best places to eat anywhere in the world now)
  • Mid-October we visited Bletchley Park. Nice.
  • In late October we (Ernest Fontich and myself) submitted our paper Normal forms and Sternberg conjugation theorems for infinite dimensional coupled map lattice. Now we need to wait.
  • In November we visited Brussels (Ghent and Brugge too), and took an unofficial tour of the European Council with a friend who works there.
  • In December I attended for the second time Scala Exchange, and the extra community day (excellent tutorials by Heiko Seeberger and Travis Brown). Was even better than last year (maybe because I knew more people?) and I already got my tickets for next year.
  • In December we attended a wine and cheese pairing (with Francesc, our man in Brussels, and Laia) at Parés Baltà. They follow biodynamic principles (no herbicides, as natural as they can get, etc) and offer added sulfite free wines, too. They are excellent: neither Laia nor I drink, and we bought 4 bottles of their wines and cavas.
Last year I decided to start contributing to open source software this year, and I managed to become a contributor to the following projects:

I wanted to contribute to the Go compiler code base, but didn’t find an interesting issue. Maybe this year.

Learning


This year I didn’t push courses/learning as strongly as last year... Or at least this is what I thought before writing this post.

  • In August I took Apache Kafka Series - Learn Apache Kafka for Beginners, with the rest of the courses in the series waiting for me having more time available.
  • In September I tried to learn knitting and lace, but it does not seem to suit me.
  • In September I enrolled in a weekly Taichi and Qi Gong course by Mei Quan Tai Chi. Will repeat for the next term
  • In December I started learning about Cardistry

Reading

I have read slightly less than last year (36 books vs 44 last year), and the main victim has been fiction. Haven’t read much, and the best... has been the re-read of Zelazny’s Chronicles of Amber. Still excellent. I have enlarged my collection of Zelazny books, now I have more than 30.

As far as non-fiction goes, I have specially enjoyed:
  • Essentialism: given how many things I do at once, this book felt quite refreshing
  • Rich dad, poor dad: Nothing too fancy, just common sense. Invest on having assets (money-generating items) instead of liabilities (money-sucking items, like the house you live in)
  • 10% Entrepreneur: Links very well with the above. Being a 10% entrepreneur is a natural way to invest in your assets.
  • The Checklist Manifesto: Checklists are a way to automate your life. I have read several books around this concept (“creating and tweaking systems”, as a concept) and it resonates with me. If I can automate (even if I’m the machine), it’s a neat win.
  • The Subtle Art of Not Giving a F*ck: Recommended read. For no particular reason. I’ve heard that the audiobook version is great, too.

Music/events


This year I have listened mostly to Sonata Arctica. We attended their concert in Glasgow (March) and it was awesome, they are really good live. This was a build up for KISS at the O2 in London (May) which was totally terrific. And followed by Bat Out of Hell (opening day!) in London. It was great, and probably the closest I’ll ever be to listening Meat Loaf live. Lately I’ve been listening to a very short playlist I have by Loquillo, and also Anachronist.

We have also attended a performance by Penn and Teller (excellent), and IIRC we have also watched just one screening: The Last Jedi (meh, but Laia liked it).

Gadgets

This year I have gotten hold of a lot of gadgets. I mention only the terribly useful or interesting
  • From last year, iPhone 7 “small”. Not happy with it. Battery life sucks big time, I got a external Mophie battery for it.
  • Mid-year: Apple Watch Series 2. Pretty cool, and more useful than I expected.
  • Late this year: AirPods. THEY ARE AWESOME
  • Laptop foldable cooling support. While taking the deep learning course my Air got very hot, and I needed some way to get it as cool as possible.
  • Nutribullet. My morning driver is banana, Kit Kat chunky, milk, golden flax seed, guarana.
  • Icebreaker merino underwear. I sweat a bit, and get easily chaffed on the side of my legs (where it contacts my underwear). Not any more: not only is wool better at sweat-handling, but the fabric also feels better on the skin. And not, does not feel hot in the summer.
  • Double Edge Shaving. I hated shaving (and actually just kept my beard trimmed so it was never a real beard or a clean shave...) and this razor (not this one specifically, safety razors are pretty much all the same) has changed that. Now I shave regularly and enjoy it a lot (together with this soap and this after shave balm)
  • Chilly bottles. They work really well to keep drinks cold or hot. I’ll be getting their food container soon.
  • Plenty of lightning cables. You can never have enough of these. I also got this great multi-device charger, ideal for traveling.
  • Compact wallet. I’ve been shown the ads so many times I finally moved from my Tyvek wallets to one from Bellroy. It is very good.
  • Book darts. Small bookmarks that don’t get lost, look great and can double as line markers. Also, they don’t add bulk to a book, so you can have many in the same book without damaging it at all. They are great, I’m getting a second tin in my next Amazon order of stuff.
  • Two frames from an artist I saw showcased in our previous office (they had exhibits downstairs). Blue Plaque Doors and Hatchard’s, by Luke Adam Hawker.
On the fun side, I also have a spiral didgeridoo, a proper Scottish bagpipes, a Lego Mindstorms I have not played with yet :( and an Arduboy. Oh, and a Raspberry Pi Zero Wireless.


Written by Ruben Berenguel

20171119

Want to play? Get a MFi controller for Christmas

It would probably be an understatement to say I'm not a gamer. Last console I got was a Game Boy, 22 years ago (IIRC). Never got a gaming PC. I have 4 games on my Mac, and one is ASCII based. But my main source of mindless leisure for the past 7 years, aside from reading, has been playing with some iDevice. From an iPod Touch back then to my current iPhone and iPad Pro, playing, for me, has been Apple. Mostly strategy games, occasionally getting some simulator or arcade depending on how good it looked and how well designed for touch it seemed.

Recently, though, I got a MFI (Made for iDevices, I guess) Bluetooth controller (affiliate link to Amazon: SteelSeries Nimbus). It has made playing the games it is compatible with impressive. Case in point, yesterday I finished playing the outstanding Forma.8 Go an open world exploration-driven platformer. I had started playing it several months ago, and stopped by the frustration at my inability with touch controls. Mind you, they are very well designed, it's just that I could not adjust to them. Playing with a controller was a totally different experience, much more enjoyable (looking at you, Downwell).

It is rechargeable (have been using it for a couple of weeks and so far the battery has not depleted... as I said, I'm not a huge gamer) and works with iPhones, iPads, Apple TVs and even with Mac's, although for older games you will need several tricks (first, a converter from MFi to HID you can compile from this github repository, and depending on the game a HID remapper tool like Enjoyable, with these two I could almost play Reassembly... I will try to play following another path, but with this I could play old DOS based games instead).

Nimbus SteelSeries MFi Controller
The SteelSeries Nimbus

Sadly, not all games support controllers, they have to be specially programmed for it. For the ones that support it though, playing with a controller is way more fun, which has made the purchase totally worth it, even if I play less than 2h/week on average. Be sure to check which games are available (and which you already own and like!) before getting it to avoid a letdown. Oh, and in case you are wondering... neither PES 2018 nor FIFA 18 support it, sadly. NBA 2K18 seems to do, but I have not purchased it yet to try. Some of the GTA titles do, as well as the iOS classics Thomas was Alone and Galaxy on Fire 2.

Oh, and in case you prefer different kinds of controllers, there are plenty to choose from. Some convert your iPhone in what looks like a weird PSP, or have a clip to hang your phone. Can't vouch for them: the Nimbus feels high quality.
Written by Ruben Berenguel

20170820

Change the parameters of a docker container without knowing the docker run command used

I'm not sure how useful this Docker "trick" is, since it happens in a very niche situation.

At work, we have several instances running a suite of Docker containers, with some non-trivial amount of environment variables, port configurations, volumes and links among them. This is set up really easily using ansible: writing link/port/volume mappings in ansible (using the docker, container or docker-container modules, depending how long ago we set it up). The syntax is similar to docker-compose, but scripted for ansible.

The problem arises when you are logged in on one of the machines and you want to change (or add) one parameter to one of the currently running containers. Re-running the ansible script in full is not always an option (for reasons I don't want to get into), so up until now we had to painfully rewrite the docker run command from the ansible set up, by hand. Which is terribly error prone. And a pain. And in general very much a process you don't want to follow, ever. We were very close to writing a short python script to convert the yaml specifications from python into docker run commands when we found out a relatively less painful way of doing it.

We jokingly call it the Indy swap



It's quite easy, and is best described using a checklist to avoid messing up.
  1. First, you figure out what you want to do. For instance, map a port
  2. Change to super-user
  3. Get the container id from docker ps
  4. Use your favourite remote editor to inspect the file /var/lib/docker/containers/{container_id}/config.json to figure out what you need to add
  5. service docker stop
  6. Edit the file in step 4 to add/remove any extra setting
  7. service docker start
  8. Voilà

Steps 5-7 are the actual Indy swap, if we are not fast enough the healthcheks in our elastic load balancers will mark the instance as failing and stop it. Yes, we could disable the check while we do this, but where's the thrill, then?

By the way, the file in 4 will be a JSON file with the settings of the container. Any saved edit needs to be done with the docker service closed (since the file is in use otherwise). The format of adding ports, volumes or links is pretty self-explanatory as long as you have already a port, link or volume set up in the instance. Otherwise, check the configuration of any other running container having that. I suggest opening it in 4 with the editor already so you can up-arrow just after stopping the service without needing to find it, and also having planned the changes beforehand (or having done them in a config.json1 you then swap)

Oh, and don't do this in production. Oh, of course you won't because you are not using Docker in production, are you?
Written by Ruben Berenguel