Hacking together a VCR for JavaScript

At SimpleReach we have a very large number of internal API endpoints and each one can bring back dozens of fields of data into our Ember.js application. In order to make integration tests we started with static fixtures, but very soon the sheer number of fields and endpoints become overwhelming.

I really wanted something like VCR for Rails, but alas, there is nothing in the front-end world that does that yet.

Chrome provided a great hack to use though. In the network developer tools you can right click and select “save as HAR with content”. This will save a recording of all network requests that have happened since you opened the tool. Voila, instant tape of all the network requests that have happened on a site.


Once you have the HAR saved, you need a way to serve the information up programatically. Sinon.js has a mock server that will respond with saved JS based on incoming requests. Setting it up looks something like this:

Now, as long as an object has been passed to “server.respondWith” the server will respond to that.

In order to make the HAR file play nice with Sinon, I forked an application called “Mockbuilder“. Mockbuilder allows you to add a HAR file to a folder and it will automatically convert it into something that plays nicely with Sinon. The fork allows you to specify a url for internal request overwrites and a location where the new mock file should go. That way you can have it send them immediately to the mock folder in your application.

Not only does this make writing JavaScript integration tests much easier, it is also a great way to make completely offline demos of your application. Simple click around, save the HAR file, and then with the new mocks in place, you can click around the same way and it will use the recordings instead of the actual requests. Perfect for sales teams who want to have a gurantee that things will work the way they expected.

Presentation below.

EmberConf Presentation: Using Ember to Make the Seemingly Impossible Easy

At this year’s EmberConf on the 25th of March I gave a presentation with Heyjin highlighting some examples of times where using Ember has made our jobs seem infinitely easier.

EmberConf was an amazing experience, the Ember community is full of genuine and welcoming people and I am so excited to see the great things that this community will build together.

Slides and video below.

Ember.js NYC presentation: Your front end is not your back end

Last month I gave a lightning talk at the Ember.js NYC meetup. The talk was about designing your API based on what it was going to be used for by the users instead of designing it based on what is in the database.  I think that this tactic can be really effective as it reduces the complexity of the front end code, which is where your users are and where you really want everything to revolve around their experience, not the intricacies of your data layer. With more and more complexity living in the front end through frameworks like Ember.js our design decisions need to be giving more weight to the front end than they used to.

Video and slides below:

 

Video:

Slides:

http://slid.es/andremalan/your-front-end-is-not-your-back-end/

http://slid.es/andremalan/your-front-end-is-not-your-back-end/embed

Coding Reflection – Reading and Testing

Reading

Technology booksReading books about the technologies that you are using is really important. It gives you something that just looking at existing code doesn’t. Over time a code base tends to train developers into doing things “it’s way” and it takes an external influence to help pull the team into better ways of doing things. My Kindle has been an amazing resource, as I can have all of my technical books with me whenever I am on the train or find myself waiting in line. Then, when I’m working at my computer, I can use the Kindle app to pull up relevant sections that I remember. Searching for something that you know exists in a good technical book can be 50 times faster than looking for it on Google.

One important optimization to this is to spend time reading books about the things that you are doing. And not just “oh, I’m writing rails code, so I should read a Ruby on Rails book”. That’s a great place to start, but if you have a specific thing you will be  working on the next day, read that book. So if I was going to spend my next day writing and fixing tests, I read an RSpec book. If the stuff that I’m working on is mainly plain Ruby code, I read a Ruby book. Playing with Routes tomorrow? read the Routes section of a Rails book.

Testing

Unit testing is unit testing, not integration testing. Don’t confuse them. Tightly coupled unit tests where you create factories for 6 objects and rely on pulling in RSS or JSON from an external service make life hell for everyone. Yes, it’s more work and it seems silly to mock everything out when you’re doing it, but being militant is the only way to keep things clean. It also makes the specs a lot faster, something that makes up for the extra time you spend building a mock object instead of just using FactoryGirl.

The other benefit is that if your proper unit tests are hard to write then that’s a pretty clear code smell. Rewriting the code to make the testing easier can lead to much cleaner, less coupled code. When you have to pay attention to everything that your code interacts with (by mocking out each interaction), you become a lot more sensitive to how much it relies on that other stuff that it really shouldn’t be relying on.

 

 

 

Reflection Reboot

printer

In my last coding reflection post I spoke about how awesome the reflection process was and how much it was becoming a habit. Seeing as that was over 2 months ago… I guess I spoke too soon.

I did do my reflections for a week or so after that, but didn’t post them. The reasons for stopping were two-fold:

1) I was running out of “easy” reflections and had to dig deeper to get there.

2) I started Taekwondo. While Taekwondo is awesome (I’m happier and fitter than I have been in a long time) it means that instead of coding till I can’t anymore, then reflecting for a bit and going home, I now work up until practice starts (my Dojang is a block away from Red Rover HQ making it easy to work until the last possible minute). Practice then clears my head of the code (which is great… it stops the nightmares) but it also clears my head of reflective thoughts on that code.

So noting that in the time that I was reflecting my skill level went through the roof, I need to change the system so that reflection once again becomes easy.

The plan is:
ReBoot Discs

1) Instead of reflecting at the end of the day, I will do it at 5pm every day.

2) Instead of making the expectation: “think of all the things you have learnt today” I’m going to reduce the load to: “think of one thing you’ve learnt” and “create shortcut that you could have taken” (taking a cue from Yan).

3) Posts aren’t going to be numbered anymore, but themed. That way I can make each one more self-contained and coherent, while again reducing the pressure to put in too much.

4) I’m going to keep a page of “rules” that I can edit and update as I learn things, to make it easier to quickly review and scan my most importnat learning in the morning.

Coding Reflections – Week 3

Week 3 of attempting to speed up my learning process through reflection.

On the reflection process:

After 15 days of practice, this is really starting to feel like a habit. I’m starting to see the circles get smaller (the Art of Learning is a great book) in my work and when previously I could only think about things in their macro forms, I am discovering even more subtleties.

One way that I am starting  to make the circles smaller in the reflection process is by forming the habit of reflecting immediately when something happens. I think it has to do with what this article suggests, that  the habit of reflecting on mistakes as they happen can greatly increase the speed at which one learns. So instead of spending a long time at the end of the day thinking back on what I did, I am instead reflecting at the point of the mistake (which for me usually means taking too long to find a bug) and trying to find ways to avoid it in the future.

 

Programming:

Sometimes doing the smallest thing doesn’t mean starting small. Instead of building up to something big piece by piece, it can often be more effective to take a large block of code that currently works (from somewhere else in the code base or the internet) and whittling that down to only the essentials. There is skill required in making sure you got rid of all the non-essentials, but if you’re diligent, it can be really successful.

 
concentrate on differences. When faced with a complex system that works and a simpler one that doesn’t, you can either increase the complexity of the broken system until it works, or decrease the complexity of the working one in order to see what parts are truly necessary for what you need done. An example is that this week I had a component that when given a hash with 10 entries would work, but that wouldn’t work when I fed it my own hash with 3 entries. I then methodically added things to my hash until it was exactly the same as the complex one and the function still didn’t accept it. Finally I went the other way and stripped the 10 item hash down to 3.. and the function still accepted that hash. Now with much less complexity I could look deeper and see that the difference wasn’t in the entries, but in the hash type (the one that worked was a “HashWithIndifferentAccess” instead of a normal Hash)

 

Look for more than one example. When you are working with a large code base and have to do something that is done somewhere else in the application, look hard for other places where things are used. Remember to check more than one place instead of assuming that the code is DRY and it’s all done the same way throughout the application. If you do find that there is more than one way to do something, be sure to use the best way possible and then go back and update the bad example.

Kitteh used Scratch!!  Its super effective

 

Write things down. The tactic of writing out all the possible causes has been proving really effective. Like super effective! I’m constantly amazed at how well it helps me spot the areas where I should be focusing on. Tracing out the actual flow of a piece of code across functions and source files is way better out in the open than in your head, your mind makes false leaps and assumptions that the cold hard light of a whiteboard marker uncovers easily.

 

Code in flow state more often. I’ve spent a lot of time building my trigger… now I just need use it more often.

 

 

 

 

Coding Reflections – Part 2

Continuing with my attempt to increase the speed at which I master the craft of programming, here is my second week of reflection.

One part of the reflection process that I noted was that reminding yourself of the previous reflections is crucial. Toward the end of the week I found that rereading me reflections from earlier really helped me to not repeat those previous mistakes.

Week 2:

Always make the minimal amount of changes to go from working to not working. Yes, I know i started with this one last week, but it bears repeating. This week I did a lot of building new things and again found myself being slowed down by making too many changes and then having to figure out which one of those changes didn’t work as they were supposed to. Starting with the simplest thing takes discipline. I’m not sure why that is, but our brains seem to want to see big effects and use those to trip us up. In order to practice the simplest thing we need to be continually vigilant.

Pay careful attention to any error trace. Often the answer is buried deeply and even though after making changes it may look like the trace is the same as before, there could be changes in where the error came from a few lines down from the top of the trace.

Write bad code. One problem that I’ve been having (probably magnified by the fact that I’m working  in Ruby on Rails) is that everything I read really concentrates on writing and architecting really good code. The problem with that is that it is really hard to know what the best code is as you go along. As you code you get paralyzed by the different design choices. You then spend hours trying to make what you think may be good design work, only to throw it away because the direction of the code means that another design pattern actually makes more sense. Instead, if you write crappy code that just gets the feature complete as quickly as possible, you end up with a complete feature and time left over to refactor (of course this only works if you’ve written tests to ensure that everything stays working after the refactoring is finished). I think it’s a lot easier to see where to fix bad code than it is to ponder over what the best code is to write. Of course, a big part of this strategy is the commitment to go back and actually do the refactoring. Code review helps a lot as sheer embarrassment can be a great motivator for going back and making sure your code looks good!

Danbo conoce a Domo - Danbo meets Domo

 Follow the Law of Demeter. There are many complex aspects to this law, but the heuristic that I’ve been using is thinking about it as “don’t take your toys apart”. If your context has a class, then you can call any of it’s methods or attributes, but you cannot “break the toy apart” and call methods of that toy’s attributes. This article does a good job of explaining why breaking the Law would seem silly in a real world context. When you buy something, when the cashier asks you to pay, they don’t “break you apart” by grabbing your wallet (your attribute) and taking the money straight out. Instead, they ask you for the money and you give it to them. They don’t even have to know about your wallet, maybe you don’t have one and just shove your money into your pockets, that would make a waiter trained to take money from wallets very confused and would be a very awkward end to the evening. Throughout the week I found that code following the law was super easy to debug and refactor, code that didn’t wasn’t, with fun little errors popping up like “you called x on nil”.

 

Coding Reflections – Week 1

Working full time as a developer again, I’m trying to apply my years of thinking about learning to my programming. Learning in school is one thing, it’s supposed to be the point of school after all. Learning at school became even easier for me once I reached the all important “learning matters far more than grades” conclusion. Learning while working full time is very different though. You have to explicitly decide to learn, to take extra steps to learn, otherwise learning stagnates and you find yourself stuck at the same skill level for years. I’m determined to not let that happen, so will be making every effort to keep up the pace of my learning.

Of course, there are many ways to practice “continuous learning”, but the one that I’m going to be attempting is just reflection. Sure, learning “the hard way” and other techniques may be more effective in the short run, but using them while at the same time producing as much good work as possible is hard. Using reflection as a tool for learning has the benefit of not taking time away from other important tasks and is arguably a more sustainable learning habit to create.

So the learning plan is as follows:

Every day after work, I reflect on what happened. Where did I make mistakes, where was I brilliant? what did I learn?

At the end of every week I can review the learning and summarize it. I will try and post these summaries here every week. Some of it will be new learning, some of it will be things I’ve forgotten some will be things I didn’t do because of lack of sleep and some will be pure guesses as to what will help make things better. I will break things into sections, general programming  and language specific learning.

Week 1

General Programming:

Much of this week was spent refactoring old code, in order to make a new set of features easier to implement. The process took me much longer than it should have. Here are some ways that I think can make the process of refactoring less painful.

Always make the minimal amount of changes to go from working to not working. My method was to first change the main attributes to be named correctly, then to watch for where things broke when they called for those attributes. Terrible idea.

The last thing that one should do is rename badly named variables and attributes. The first thing to do should be to DRY up the code as much as possible, only after everything is working and the code is as elegant as possible, should you worry about renaming old bad names. a “rename_column” migration is a hell of a lot cheaper than hunting through bad code for all references to a badly named variable/attribute.

On the subject of renaming attributes never use “find and replace all”. Check each instance of something that gets replaced. If there are too many of them, then it means that you haven’t done the DRY work that you should have first. If you do use “find and replace all” then every bug that you encounter for the next hour should have you thinking… “could this have been because of that find and replace all that I shouldn’t have done?”

Basic debugging:

Resist the temptation to just mindlessly trace variables when there is a problem. After the first or second tweak to try and make things work, stop and take a breath. Start enumerating the things that could be wrong. Don’t trust your brain to hold all of these elements, write the various causes for the error down (yesterday I stuck a large strip of dry-erase roll on my des to help with this). Doing it without looking at the code too much helps as well, it gets you thinking more abstractly and actually allows you to put those years of listening to Computer Science lectures to use. Think of all the different ways that you can actually test to see which one of these causes it is. Once you have a list, rank them by probability and work from there. As Steve Wolfman told me on my very first CS lecture at UBC, “Computer Science is the science of clear thought”. Writing things down is the easiest way to make your thoughts clear.

 Ruby on Rails:

use validates_presence_of to ensure that variables that you need are there. Make sure you actually need those variables.

The before_validation callback can be really useful to consolidate and double-check that everything your model needs is there.

Spend every effort as a team to make the specs fast. Every extra second that a test takes, gets multiplied by the number of developers running tests by the number of times it’s run over it’s lifetime. Those extra seconds can easily add up to hours on a big complex project. Slow specs lead to less testing which leads to broken code.

Single table inheritance in Rails is easy. It can be done simply by making a class a subclass or another ActiveRecord::Base class. The caveat is that that superclass’s table needs to have the “type” attribute, in which Rails will automatically store the type of the subclasses.

When converting a non-ActiveRecord model to an ActiveRecord model pay careful attention to methods that may be overriding ActiveRecord methods. Deal with those first.

Rails AntiPatterns is a really good book. I’ve been trying to force myself to read and watch as much as possible on programming in Rails and this book has been by far the most engaging!

Themes – A personal journey

Twenty hand

My site has gone through a lot of them changes over it’s history. Even though one of the bloggers that I admire most (at least before he went off the deep end) disagrees with theme changes, I feel that creating and changing my themes has allowed me to flesh out my ideas around aesthetics as well as my sense of self and personal style. I started university believing that I had no artistic talent whatsoever and have slowly come to realize that I just never spent any time developing it.  I treat my theme as a personal journey, it showcases my knowledge, ability and feelings at a given point in time and allows me to show everyone in a visual way when those things change by updating my theme.

So although I’ve lost a few of the steps along the way, here is a subset of the themes I’ve hacked along the way…

New Theme: twenty-hand

I’ve been unhappy with my theme for the page few months and not had the time to change it. I’ve finally got around to doing so and at the same time have played around with some WordPress 3.0 goodness. Here are some of the features:

  1. It’s a child theme of the new WordPress theme twentyTen.
  2. It’s much cleaner than my old theme.
  3. It uses the new wp_nav_menu system. The 7 menus that I add are the 7 menu items at the top.

Continue reading “New Theme: twenty-hand”