Are we giving users a loaded gun?
On one of my projects, a feature we implemented gave a select group of users of the application the ability to curate content that appears on the site by inputing something using a proprietary query language. The sources of that content could be coming in externally and they needed to be in an authorized list before we were allowed to display it on our site. This list is maintained in the code and used when we do a site wide search for content, but since the users were now able to type in a query of their own, we were assuming that they knew what was authorized and what wasn't.
However, a showstopper bug was filed later that said that the site was displaying content that it wasn't authorized to, and that we could be sued by those content providers because of that.
A few solutions to the problem that I can think of:
1. Educate the users on what can be inputed and what can't.
2. Don't trust that the users know what they're doing and compensate for that behavior in the code by stripping out what they entered and only using it if it was valid.
3. This is similar to 2: Build in a validation tool and don't allow users to save their queries unless it passes validation.
It's a very frustrating situation to be in, especially when it's not because there's a problem in the code but in the way the software was used.
Band: Black Dub
I think this newcomer to the block deserves a mention, not entirely because of its star lineup but because their music hit me immediately after I heard it. Frontman Daniel Lanois is a Canadian producer who is the man behind the great names like U2, Bob Dylan, Neil Young and Peter Gabriel to say the least. In October of 2009, he started his own project: Black Dub with Brian Blade on drums (the genius behind Brian Blade & The Fellowship Band; he also collaborated with Chick Corea and John McLaughlin briefly on the Five Peace Band, replacing Vinnie Colaiuta on a few venues of their tour, namely in Sydney where I saw them live), bassist Daryl Johnson and the soulful, multi-instrumentalist Trixie Whitley, daughter of the talented but relatively un-mainstream late Chris Whitley.
One can probably describe their music to be rock-infused blues, but with the presence of the likes of Lanois, Whitley and Blade, you will most definitely hear more than that, jazz included. Here's a video of them going unplugged at the NPR offices.
Git: Rebasing tips
Git allows you to apply a rebase instead of a merge, which gives a cleaner set of commits and history. See git-rebase explained briefly here.
Chris and I ran into a few issues with rebasing this past week and I thought I'd share our learnings.
We created a fork of the master branch of jorgej/RapidFTR here: dlbock/RapidFTR and had to move our changes back to jorgej.
Lesson #1
Never do a rebase unless you're ready to push all changes to the master remote/branch that you forked/branched from. So if you've done a rebase, never continue pushing to your branch/fork - because when you have to do a merge or a rebase again in the future you will see the commits that were rebased previously more than once.
Lesson #2
Instead of doing a rebase by trolling through your commits one at the time, you could squash all your commits into one and perform the rebase with just that one commit that you cherry-picked.
Let's say that the last 4 commits from your local repository's HEAD are the ones you've done on top of jorgej/RapidFTR.
git rebase -i HEAD~4
This will output:
pick ca6e97c Fixed bug where it wasnt stripping all non a-z 0-9 chars and also got workng in IE6 pick 5b6964c Fixed server side function to remove non alphanum characters from field name when getting it from display name pick 7e68448 ... and fixed the javascript function pick 15483c6 First of IE fixes. Apparently IE sometimes sends blank accept header which was causing app to go into api mode. Now only does it when accept type is xml or json # Rebase e2d88ea..945647e onto e2d88ea # # Commands: # p, pick = use commit # r, reword = use commit, but edit the commit message # e, edit = use commit, but stop for amending # s, squash = use commit, but meld into previous commit # f, fixup = like "squash", but discard this commit's log message # # If you remove a line here THAT COMMIT WILL BE LOST. # However, if you remove everything, the rebase will be aborted.
Pick the commit you want, squash the rest and then save this.
pick ca6e97c Fixed bug where it wasnt stripping all non a-z 0-9 chars and also got workng in IE6 s 5b6964c Fixed server side function to remove non alphanum characters from field name when getting it from display name s 7e68448 ... and fixed the javascript function s 15483c6 First of IE fixes. Apparently IE sometimes sends blank accept header which was causing app to go into api mode. Now only does it when accept type is xml or json
Another editor will pop up to allow you to change the commit message based on the rest of the 4 commits. Once you're done with squashing all 4 commits into one, you're ready to do a rebase to jorgej. Make sure you have set up jorgej as an additional remote repository.
git remote -v jorgej https://github.com/jorgej/RapidFTR.git (fetch) jorgej https://github.com/jorgej/RapidFTR.git (push) origin http://github.com/dlbock/RapidFTR.git (fetch) origin http://github.com/dlbock/RapidFTR.git (push)
To add a remote repository:
git remote add jorgej https://github.com/jorgej/RapidFTR.git
Finally perform the rebase from jorgej:
git rebase jorgej/master
Dealing with change
This week marks the movement of my blog from the free hosted wordpress.com version to one with my own domain name using wordpress.org. There were a few glitches in getting it working the way it should, but if you're reading this means that you have migrated with me to www.dahliabock.com/blog.
This week also marks my re-entry back to ThoughtWorks. Surprised? Don't be. May there be many more good years with these people I call family.
Refactoring for the refactoring's sake
Imagine for a minute that you have inherited a legacy code base (According to Michael Feathers, a legacy code base is one that is largely untested, if at all) but also have a whole list of new features that you need to build upon the existing code with a deadline. What would you do?
Available options:
1. Rip the code apart and start the cleaning process before adding new features.
2. Pile the code for the new features on top of the existing ball of mud.
3. Is there a 3rd option?
The issue with Option 1 is that it might be a while before we can deem the code base ready for development of new features, and what is the proper acceptance criteria for refactoring stories anyway? I don't think Option 2 should be an option at all, because that would just mean we're shooting ourselves in the foot, screaming in pain, and doing it over and over again. I learned recently that another thing we could do is to tack refactoring on top of new feature stories, which means whatever existing code that you touch to make things work for the new stuff that you're writing, you should stop and clean it up and write tests for it. The problem, as with all refactoring tasks, is when do we stop? I don't think I've found a good answer to that question just yet, but just that it should be decided by the team.
Updated: Managing multiple Ruby versions on OS X
Turns out there's an easier way to manage the multiple ruby versions on OS X with Ruby Version Manager (rvm).
1. Follow the rvm installation instructions here. I used the recommended installation from the GitHub repository.
2. Install the version of ruby you want:
> [dahlia@rogue ~]> rvm install 1.9.2
3. Setting up your default ruby version.
-- Find out the list of rubies that you have installed:
> [dahlia@rogue ~]> rvm list
rvm rubies
ruby-1.9.2-rc2 [ x86_64 ]
-- Set the default version of ruby:
> [dahlia@rogue ~]> rvm --default ruby-1.9.2-rc2
-- Verify that it was setup correctly:
> [dahlia@rogue ~]> which ruby
/Users/dahlia/.rvm/rubies/ruby-1.9.2-rc2/bin/ruby
4. And you're done!
Hope this helps.
Setting up svn update in CruiseControl
It's been a while since I've had to set up CruiseControl from scratch because most of the time either the project is so deep into quicksand that continuous integration is the least of their problems, or there already is a CI server up and running, or the team has decided to use another CI tool.
One issue I had while getting it up and running is configuring CI to do an subversion update before doing a build. Make sure that the cruise config file has the following lines in a specific project:
<bootstrappers>
<svnbootstrapper localWorkingCopy="projects/${project.name}" />
</bootstrappers>
<modificationset quietperiod="0">
<svn LocalWorkingCopy="projects/${project.name}" />
</modificationset>
All the modificationset tag does is only check your repository if there are modifications but does not actually update the local working copy in your CI server. The svnbootstrapper is the one that does the actual update.
The documentation for svnbootstrapper was a little confusing because it says that it "Handles updating a single file (typically build.xml and/or build.properties) from a Subversion repository before the build begins" and doesn't specifically say that it would also be able to update entire directories.
I'd be curious to hear if people achieve the same results differently.
If anyone out there is on a Java project and starting to set up a CI build, I would actually recommend Hudson. It is free, and extremely easy to setup and you don't even have to worry about xml configuration files. You'll be up and running in 15 minutes or less.
Continuous Integration 101
Continuous integration was first talked about by Martin Fowler and Kent Beck back in 1999. That was 11 years ago. But I've come to learn that 11 years is nowhere close enough to making a practice mainstream.
The basic idea behind continuous integration is that if you have a team of developers frequently checking in code to a source control system (well, hopefully you do that), you need a way to verify that the code that has been checked in works seamlessly as a whole. What better way to verifying that than after every commit? And what better way of doing that verification after every commit than automating it?
A typical flow of a CI build is as follows:
1. Compile source code
2. Compile tests
3. Run tests ----> A lot of teams miss this step and what you end up with is just a continuous build which gives you no benefit at all.
4. Package deployable artifacts
(Sometimes these steps include automatic deployment to a test server(s) so that other members of the team like QAs, BAs or customers can play with the application, but that's a story for another day.)
There are many CI tools out there that will help you achieve this goal: Hudson, CruiseControl (CruiseControl.NET, CruiseControl.rb), Bamboo, TeamCity, etc. My favorite of those tools is Hudson because it is extremely easy to set up and you will be up and running in about half an hour, and it's free.
So why is Continuous Integration a valuable practice?
- It verifies your code works as a unified whole, that is if you have tests and run them. It is much more critical that integration is done sooner and continuously when development is spread out over separate teams.
- It produces a deployable artifact for someone to actually see working. This eases the effort to doing frequent deployments and opens up the possibility of automating that process as well.
Book Review: Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman
There are some books that I wish were available 4 years ago when I first started working in the software development world, maybe even before I went to college. This book is definitely one of them. It's not an easy task to concretely outline the practices that are helpful towards improving one's skill as a software developer but this book does it well as Dave and Ade speak about their experiences as apprentices and how they grew to be journeymen.
I will point out the patterns that resonate the most with me and my quest to be a better developer.
1. Expose Your Ignorance
Context of the problem: Your team/clients/managers are expecting you to know what you're doing and deliver value to the team but you're unfamiliar with some of the required technologies or the domain of the project.
This situation happens to me on quite a regular basis every time I switch to a new project. It's one thing to not know the domain of the application, it's another to not be comfortable or familiar with the language or tools that are used on the project. When I started my first project at ThoughtWorks, I remember taking about 3-4 months to get up to speed and learn all the right questions to ask when working on a story. I have reduced that learning curve period since then, but it has always been a challenge to have to deliver on my first day on a new project.
Suggestions from the book:
- Show your team and managers that the learning process is part of delivering software.
- Get used to learning. Craftsmen will have to work with different technologies and domains over the course of their career. Those who stick to just one domain/technology are called experts. Expertise is not the goal of the apprentice.
- The most important trait of a craftsman is the ability to learn, identify an area of ignorance and working to reduce it.
2. Confront Your Ignorance
Exposing your ignorance will be of no value if you don't do anything about it. There will always be tools and techniques that you need to learn and master and you have to start somewhere.
I usually do the following:
- read documentation, APIs and FAQs
- if it's open source, attach the code so it's easier to look at what's going on
Things I should really do more of:
- write a scratch application to use the new tool/library
- actually dive in to the code, if there is any
Things I am currently trying to do:
- start from scratch and learn about a language from the ground up. I recently picked up the book C# In Depth by Jon Skeet to learn about the innards of C# starting with C#1 onwards to C#3. I also was reading Programming Clojure by Stuart Halloway (but sadly that has been pushed to the wayside).
3. Be The Worst
Some people find it discouraging when they realize that they are the least experienced person on their team. I, however, find that exhilarating because I know the opportunities for me to learn will be endless. Being the worst on your team shouldn't disheartened you, but it should push you to work harder than anyone else because the goal is not to stay the weakest, it is to work your way up.
Enthusiasm and passion can be contagious and I have my coworkers at ThoughtWorks to thank for always pushing me to learn more and be better, either explicitly or implicitly.
4. Kindred Spirits
The journey to becoming a master craftsman can sometimes be a lonely one depending on the people who work with you and the work culture that you're in. It is very disheartening if your efforts to learn new tools, or languages or platforms isn't supported by your manager just because "it's the company mandate that we only use X, Y, Z products and open source tools aren't allowed". You need to surround yourself with people who are like-minded about learning and that can be achieved via user groups, mailing lists, etc.
My kindred spirit is in the form of Mark Needham, who is a friend and a coworker at ThoughtWorks. We discuss about everything from what's the best build tool to use in the .Net world, to why is it so hard to change the way people work. He is constantly challenging me to explore things further like diving into the code for Rhino Mocks to understand how things are done behind the scenes.
5. Expand Your Bandwidth
You have built up a set of skills. You are comfortable doing what you're doing now and hope things doesn't change so you don't have to set foot in unknown territory again.
It's easy to get comfortable with what you know and ignore what you don't. But unless you want to do the same thing over and over again for a long time (which is quite impossible in software development these days), there will always be new languages to learn, new tools to use, new methodologies to get used to. I don't think we can afford to rest on our laurels in the software development world. Things change so frequently and there is always so much to learn.
Tip: Read books, read blogs. Read, read, read.
Other patterns that I think are useful:
6. Breakable Toys - use your favorite tools to build a simple application of your choice. Most work environments don't allow for failure and experiments so you need to create a safe environment where you can do just that.
7. Use The Source - Seek out code and read it. A lot of the tools we use nowadays are open source so the code is readily available.
8. Record (and Share) What You Learn - I use this blog as a tool to record my learnings and as a platform to share them with others and to solicit feedback from them.
I highly recommend this book to anyone who is seeking to learn how to improve their skills as a software developer.
Feedback: Now is better than later
In his book Extreme Programming Explained, Kent Beck says:
Extreme Programming (XP) is about social change. It is about letting of habits and patterns that were adaptive in the past, but now get in the way of us doing our best work. It is about giving up the defenses that protect us but interfere with our productivity. It may leave us feeling exposed.
Whether we like it or not, change is inevitable. One of the ways of dealing with change is practicing rapid feedback. There are a few things we can do to obtain as much feedback as we need, as often as we need it.
1. Fast unit tests
Yes, I'm a nazi about tests. But I believe and know for a fact that they are my first line of defense when it comes to creating working software.
2. Continuous integration
An addition on top of automated tests is a build that runs at every checkin, and preferably a fast build too.
3. On site customer
I want to waste as little time as possible waiting for emails to be answered or specification documents to be updated. I want to walk up to my customer and be able to ask a question and obtain an answer immediately.
4. Retrospectives
This is a practice that allows a team to learn from the mistake they made in the previous iteration, and take steps to address those mistakes so that they keep improving over time.
5. Frequent releases
Working software is the best tool for obtaining feedback. The faster you get your software out to your customers the better. That way we can adapt to the changes that are needed and progress accordingly.
Contact Me
Archives
- January 2012 (2)
- December 2011 (1)
- October 2011 (1)
- August 2011 (3)
- June 2011 (2)
- March 2011 (1)
- February 2011 (1)
- November 2010 (3)
- September 2010 (1)
- August 2010 (1)
- July 2010 (2)
- June 2010 (1)
- May 2010 (2)
- April 2010 (2)
- March 2010 (4)
- February 2010 (1)
- November 2009 (5)
- October 2009 (4)
- September 2009 (2)
- August 2009 (3)
- July 2009 (2)
- May 2009 (4)
- April 2009 (9)
- March 2009 (1)
- February 2009 (5)
- January 2009 (4)
- December 2008 (6)
- November 2008 (5)
- October 2008 (3)
- September 2008 (1)
- August 2008 (1)
Categories
- .net (3)
- active record (1)
- agile (9)
- band (1)
- books (13)
- c# (3)
- clojure (2)
- coding (2)
- coding practices (8)
- conferences (1)
- consulting (3)
- continuous intergration (2)
- error handling (1)
- flex (3)
- functional programming (2)
- git (1)
- grails (2)
- hibernate (1)
- integration (2)
- java (2)
- lean (2)
- logging (1)
- mocking (1)
- music (1)
- oop (1)
- osx (2)
- pairing (6)
- philosophy (2)
- photography (1)
- rails (1)
- ramble (10)
- ruby (2)
- Sydney (3)
- tdd (2)
- teamwork (5)
- testing (17)
- ThoughtWorks (19)
- twu (2)
- UI (3)
- web (1)