20170820

Change the parameters of a docker container without knowing the docker run command used

I'm not sure how useful this Docker "trick" is, since it happens in a very niche situation.

At work, we have several instances running a suite of Docker containers, with some non-trivial amount of environment variables, port configurations, volumes and links among them. This is set up really easily using ansible: writing link/port/volume mappings in ansible (using the docker, container or docker-container modules, depending how long ago we set it up). The syntax is similar to docker-compose, but scripted for ansible.

The problem arises when you are logged in on one of the machines and you want to change (or add) one parameter to one of the currently running containers. Re-running the ansible script in full is not always an option (for reasons I don't want to get into), so up until now we had to painfully rewrite the docker run command from the ansible set up, by hand. Which is terribly error prone. And a pain. And in general very much a process you don't want to follow, ever. We were very close to writing a short python script to convert the yaml specifications from python into docker run commands when we found out a relatively less painful way of doing it.

We jokingly call it the Indy swap



It's quite easy, and is best described using a checklist to avoid messing up.
  1. First, you figure out what you want to do. For instance, map a port
  2. Change to super-user
  3. Get the container id from docker ps
  4. Use your favourite remote editor to inspect the file /var/lib/docker/containers/{container_id}/config.json to figure out what you need to add
  5. service docker stop
  6. Edit the file in step 4 to add/remove any extra setting
  7. service docker start
  8. Voilà

Steps 5-7 are the actual Indy swap, if we are not fast enough the healthcheks in our elastic load balancers will mark the instance as failing and stop it. Yes, we could disable the check while we do this, but where's the thrill, then?

By the way, the file in 4 will be a JSON file with the settings of the container. Any saved edit needs to be done with the docker service closed (since the file is in use otherwise). The format of adding ports, volumes or links is pretty self-explanatory as long as you have already a port, link or volume set up in the instance. Otherwise, check the configuration of any other running container having that. I suggest opening it in 4 with the editor already so you can up-arrow just after stopping the service without needing to find it, and also having planned the changes beforehand (or having done them in a config.json1 you then swap)

Oh, and don't do this in production. Oh, of course you won't because you are not using Docker in production, are you?
Written by Ruben Berenguel

20170507

Shading dependencies with sbt-assembly (in particular, shapeless in Spark 2.1.0)

A few weeks ago I needed to parse configuration files in Scala for a Spark project and decided to use PureConfig. It is incredibly lean to use, needing minimal boilerplate. I recommend you check it out (give also a look at CaseClassy, which I haven't had time to test yet).
Everything seemed straightforward enough, and I got it working pretty quickly (as in, it compiled properly). The surprise? spark-submit failed with a conflict with Shapeless (lacking a witness). This is due to Spark 2.1.0 needing Breeze, which in turn needs Shapeless 2.0.0 (which is pretty old). Problem is, Spark's required library prevented PureConfig from pulling the correct version. D'oh!
There is an easy fix, though, if you are using sbt-assembly you can shade dependencies, by adding something like the following to your assembly.sbt file:
assemblyShadeRules in assembly := Seq(ShadeRule.rename("shapeless.**" -> "new_shapeless.@1").inAll)
Written by Ruben Berenguel

20161231

2016 in Review

Some of the links are affiliate links to Amazon. I only recommend what I use.

At last

February. Finished my PhD dissertation, so can add Dr. in front of my name when ordering a Gatwick Express train ticket. Also makes for a cooler email signature. So far has been the only differences I’ve seen in the year

Work

February-December Started working as a consultant in London, almost the day after delivering the PhD presentation. Not a big cause-effect relationship, though. Have done and learnt a lot of different things since then:
  • Learnt how horrible Pig Latin is for MapReduce jobs. Reference Programming Pig
  • Learnt lots of ins-and-outs of setting up single machines for services and clusters for machine learning in AWS.
  • Improved my knowledge of Spark (which was with the Python API) by learning Scala. Delivered a few small machine learning jobs in Spark/Scala. Can recommend Advanced Analytics with Spark (still reading but liking it a lot so far).
  • Wrote some production-level Go for backend services, reviewed a ton more (not the main Go developer). Got a copy of The Go Programming Language in paperback for the kicks, sits side to side with The C Programming Language I’ve had since forever.
  • Wrote a ton of production-level Python for backend services (some machine learning related, some data pipeline related, some just service related) and reviewed some little bits (acting as the main Python developer). Skimmed Fluent Python (futures, specially). Will dive into it completely in 2017.
  • Learnt to use Docker (and Ansible, even if not directly related) properly.
  • Used Apache Kafka (consumers and producers in Go, Scala and Python), Aerospike (in Go and Python) and RabbitMq (in Python)
  • Did a lot of fine-tuning of the queries, tables and key distribution of a Redshift (PostgreSQL) cluster (20% space saving).
  • Went on a machine cleaning spree on AWS, shutting down (with refactors where needed) or downscaling systems. We saved something like 30-35% of costs doing this.
  • Discovered how much I enjoy spinning tops to unfocus a few seconds and clear my mind to tackle a problem.



April-December Worked on some search engine optimisation analysis for a French company. Still ongoing, trying to get a 25% raise in organic search (right now at 10%)

August-December Started working on a personal project: an adserver for this blog, with a multi-armed bandit approach to serve relevant Amazon affiliates product ads. You can read up a little more about it in this post I wrote on LinkedIn.

Life

February-December. Spent 3 weeks out of every 4 in London.

Attended loads of meetups (member of the London Scala User Group, Spark London, London Python… you get the idea), ate loads of pulled pork burgers, duck crackling burgers, pork burgers, etc. Also visited some museums. Cafés in London museums are pretty good.

Also attended several meetups in Barcelona, when I happened to be here (Barcelona Python Meetup, Barcelona Spark Meetup, Golang Barcelona).

Became kind of a regular at the Café at Foyles in Charing Cross (awesome coconut and lime slice) and the Café at Waterstones in Hampstead (best scones I've eaten in London).

February Visited Llívia. Lovely small town, will repeat.

February Attended Lineapelle Milano 2016

June Visited Hay-On-Wye, famed by its second-hand bookstores and eggs Benedict. Will repeat. Probably kidnap Jasper Fforde to get him to write Paint by Numbers

July Took a course in wood carving in London. Will repeat.

September Holidays in Collioure. Will repeat

Should have done it sooner

June Learnt to touch type and moved to using the Colemak keyboard layout, practiced at Keybr. Makes a huge difference to speed, accuracy and ergonomy. Should have done this way earlier in my life.

June Started eating parsnip, suggested by our product owner. Never had before, I'm liking it.

September Learnt to prepare brown butter thanks to Ruhlman's Twenty. Ideal for some desserts, caramel sauces and mashed vegetables.

October Discovered I actually like sweet potatoes, specially as French fries.

Learning

March-June Earned a postgraduate degree on Big Data and Data Science from the University of Barcelona.

February-December Attended the Spark, Emacs, Scala and ScalaX Bytes meetups every time I was in town. More sparingly attended the Data Science London, London Machine Learning, Python London, Go London User Group and Apache Kafka London, among a few other meetups I only attend on a interesting-talk basis.

June Took (as in, finished some) the Coursera courses about Scala. Will finish the rest eventually.


December Attended Scala Exchange 2016 London and Scala Exchange Hack day, where I got an introduction into contributing to Open Source. Trying to help with scala-native right now. Got a T-shirt with the text Scala contributor, which is nice.

Reading

This year I read 44 books, most of them non-fiction and mostly about self-improvement and data science.

I recommend Chronicles of Amber by Roger Zelazny as fiction and Never Split the Difference by Chris Voss as non-fiction. Technical, I’d recommend The Software Craftsman: Professionalism, Pragmatism, Pride by Sandro Mancuso

Music

March-June Enjoyed the hillbilly covers of Hayseed Dixie and almost attended a concert in Canterbury

September-December Discovered Sonata Arctica and really enjoyed its songs. Specially partial to FullMoon and What did you do in the war, dad?

October Attended the Boxes tour concert of The Goo Goo Dolls in London (luckily they played most of the classics)

November Attended the first concert of the Chimera Orchestra in London

December Got these active-noise cancelling earphones for Black Friday. Work wonders to quieten a noisy office.

Gadgetry

Aside from the ANC earphones mentioned above, this year I got
  • Mechanical keyboard. Great key feel and travel, but very noisy. Got complaints in the office. Also weights a ton (and a half)
  • Portable foldable keyboard. Surprisingly good key feel and shape. Works excellent, aside from being unable to pair to more than 1 device at once.
  • Spiraliser. Great for potato spaghetti. Got it in December and only started using it in Christmas, is going to make most of my solid vegetables this 2017
  • iPad Pro with Pencil. After trying the Pencil with Procreate in an Apple Store I knew I needed one. Also, this way Laia got a not-too-old iPad Air, too. Got the largest, works great. Then I got an iPhone SE and recently (due to Laia’s Android phone dying for Christmas) an iPhone 7.
  • Space pen. I love pens. I thought I was going to give up on notebooks after the Pro and Pencil, but… Always carry this sturdy little beast in my pocket together with the excellent Canson A6 notebook. Its paper can easily take watercolour. Also practicing shorthand on it.
  • Memo bottle. I got a very compact backpack (so that I’m not that guy on the tube) and this water bottle fits perfectly when I need to bring water along. A little bit on the pricey, hipstery side but works well.
  • External micro USB disk. Oh, I love this thing. Before I used my 1TB external disk for everything (since I have a 128GB Air). Which included all git repositories, meaning that when it was time to leave (or fire drill) I needed to close iTerm, emacs, SourceTree... not anymore. I have it always plugged, effectively doubling my hard drive space.

Weird

Introduced our company into the misteries of the Catalan tradition of the Caga Tió. Brought them one for Christmas.
Written by Ruben Berenguel