How To Hire Outsourced Developers

This is Part 2 of a 2 part series on hiring developers. Read Part 1, How to Hire Programmers, if you’re hiring for full time positions. Read this article if you’re outsourcing.

Are you having Android Dev Troubles?

I offer Expert On-Demand Android Consulting that can help. Click here for more info.

I recently wrote about how to hire programmers which focuses on hiring full time employees. This post is tailored to help you learn how to hire outsourced developers. You’ll see that the two methods share a lot of similarities except hiring a contractor is a bit less involved as they are not going to be joining your team full time.

A little background … I’ve hired almost 10 remote, outsourced developers who were all overseas using the method I explain below.  I feel that while its not fool proof, its the best method I’ve encountered and I feel that it has the best ROI with the lowest time commitment. Over the years I’ve used contract employees for my business and I’ve been able to create this process as a by-product of my experiences. Sometimes it went well, other times it didn’t and sometimes it was an absolute disaster. After refining my hiring practices many times over I was able to create the process below that always returned top notch talent for outsourced developers. I hope the process below does the same you.

The Process

There are 4 simple steps and if you cut out the job posting step then you actually only have three steps to go through. The goal of this process is to save you time and money while maximizing the chance of getting the best candidate you can find. Please note, during this process you will probably need to spend anywhere from $50-$250 depending on the contractors that you put through this review process – more on that later. You will be posting a job, reviewing the candidates and selecting the top X (usually at least the top 10). At that point you’ll hire them to do a simple programming task. Once they complete it (if they complete it) you’ll be able to review the code and determine who has the best implementation and then you usually will hire that person. The steps are listed below.

4 Steps to Hiring a Good Outsourced Developer

  1. Post a Job Description a Job Site
  2. Prelim Review: Hire the top Candidates (minimum 10) for 1 Hour
  3. The Programming Task
  4. Keep the Top Candidate(s)

Step 1: Post a Job Description on a Job Site

Goal: To create an interesting job posting that will attract quality applicants

Note: I prefer to use oDesk.com as the site where I hire my contract programers so this entire article is baed around that site.

Log into oDesk.com and post a job. Make sure the title accurately explains what you want the person to do. Do not post things like “JavaScript Ninja” or “Rails Badass”. Titles such as “Senior Rails Developer” or “Wordpress Developer” seem to fare a bit better than abstract titles. The description of the job should describe what you need in detail. Do not leave this as a simple one liner such as “Write good code and deliver on time.” Inform the candidate what you need from them. If its updating an existing code base inform them of what they’re getting into. If you’re a business person who does not understand code, explain what you want the product to do or what your product already does and why you need help. Don’t spend a ton of time on this. Maybe 10-15 minutes, max.

When hiring in oDesk you can provide various options about the job. For each of these sections I tend to use the following settings:

  • Skills Required
    • This is self explanatory. If you have a Rails site, put “Ruby on Rails” and probably “HTML” and “JavaScript”, etc.
    • If its another technology, put those technologies here.
  • How would you like to pay
    • Hourly when doing open ended development such as new feature work.
    • Fixed when the exact scope is defined. Such as: Install WordPress and configure theme to look like X.
  • Estimated workload
    • This varies person to person, but I usually have them on for Part Time
  • Desired Experience Level
    • Always choose Intermediate or Expert. Choosing beginner will get you some shoddy work that will eventually cost you 2x as much to replace later.
  • Marketplace Visibility
    • Anyone can find this job (UNLESS … I’m hiring a specific person)
  • Preferred Qualifications
    • Freelancer Type: No preference
    • Minimum Feedback Score: No Preference
      • The reason for this is because there are TONS of people who are experts who have never completed a job on oDesk before. Also, sometimes people get a bad score from the only job they did … not because it was bad work but because the person managing the product was terrible at their job. Don’t count them out.
    • Hours billed on ODesk: No preference
      • Again, you could have someone who is an expert who just joined. Don’t count them out.
    • Location
      • This one can be tricky. I personally do not care where they are. But, this is up to you. I usually put No Preference. However, if you are testing the various locations of the world to find what works best for you then choose that location.
    • English Level: 5 out of 5
      • This is a requirement. A lot of contractors on ODesk will say their 4/5 but when you converse with them via email, IM or Skype/Google Hangout you’ll realize they’re actually more like a 2 or a 3. Being able to communicate effectively is key to success. Without it, you’re fighting an uphill battle.
  • Screening Questions
    • I personally don’t add these as programming jobs are quite technical and either you know how to program well or you don’t.

Now, post the job.

Wait about 48 hours and then move onto step 2. During this time you’ll start having people apply for the position.

Step 2: Prelim Review – Hire the Top Candidates (at least 10)

Goal: To hire the top candidates who seem qualified in order to take them to the next step.

You’re probably going to get upwards of 25-60 applicants for your job posting from candidates all over the world. The first thing you will need to do is trim the list down. Here’s how you do it.

  • Cut anyone whose English (or whatever your chosen 5/5 language is) is terrible and very difficult to read. This is an immediate red flag. If you can’t communicate now, how do you expect to communicate later?
  • Cut anyone who does not have the skills to do the work. e.g. – If you’re hiring for a Rails developer and the candidates does not list Rails as one of their skills cut them from the list. You want the highest quality you can afford.
  • Cut anyone that doesn’t seem to fit the job description. e.g. – If you want someone to develop a Game for you, a line of business web developer probably isn’t the best fit for you.
  • Cut anyone who is way out of your budget. Sure, that guy who’s charging $155 an hour looks great but maybe your budget is $25/hr. No point in entertaining the thought of getting a $155/hr candidate on the roster. You’ll waste their time and yours.

After 48-72 hours you’ll want to trim your list down to a minimum of 10 candidates.

You’re at the point where you are now going to spend real money. If the average wage of your applicants is between $5 and $25 and hour you’ll spend roughly $50-250 to find the right person (at most). You’ll want to hire all 10 candidates. Yes, I’m serious. However, when you hire them you will inform them that you’re going to hire them to complete a simple task that should take no longer than 1 hour. You will send them the link to the Programming Task (Step 3) so they can complete the task. This one hour is PAID to the contractor when they do the work. This shows good faith to the contractor so they do not feel like they’re having to do work for free. If you ask outsourcers/freelancers/contractors to do work for free you’ll lose ALL of the good/best developers immediately. When you pay someone a full hour of their rate they take you much more seriously.

Something interesting happens at this point in the process. An average of 50% of the candidates never continue past this step. The reason is not exactly known but I suspect it is because they simply do not know how to perform the task that is given to them. That is exactly what Step 3 is for, weeding out the candidates who can do the work from those who cannot.

Step 3: The Programming Task

End Goals: Identify if the candidate has the necessary ability to solve programming problems with code. Gain insight into the quality of code that is produced by this candidate.

You’ll notice that this step is very similar to Step 3 in the How to Hire A Programmer article with a few alterations sprinkled about. While the articles share similarities there are key differences in the process so please continue reading if you’ve already read the previous article on How to Hire a Programmer.

During the Programming Task* the candidate will be tasked with solving a real problem using the language that I specify. The coding task should not take an experienced programmer more than 1 hour to complete at most. I only pay for one hour of the contractors time. They can take as long as they want to get it done, but I’ll only pay for one hour. The goal is simple: To determine if the candidate can produce the code that is needed.

The programming challenge is the great equalizer. Not because the problem is necessarily difficult (it’s not, its easy), but because it allows you to gain insight into the quality of code that the person writes. It also answers a vast array of other questions as well:

  • Does the candidate communicate effectively?
  • Does the candidate know how to follow directions?
  • Does the candidate know how to use Git and GitHub?
  • Does the candidate follow coding best practices?
  • Does the candidate know how to code?
  • Does the candidate follow through and succeed?

I usually inform the candidate that the solution to the problem is due within 24 hours. The programming challenge is posted online on GitHub/BitBucket in a Git repo that is shared across all candidates. The same repo is used for everyone. Yes, that means that others can see the same answers as other applicants (via pull request). Thats fine because if someone wants to cheat it is easy to detect. If 3 people’s repo’s look exactly the same then the person who checked in the code first is most likely the person who wrote it. No two people write the exact same code every single time.

The repo contains all the necessary information about the task. The candidate is to follow the instructions and email me when their done. Here are a few example of some of the programming challenges I’ve created:

Feel free to fork these and save them for yourself. If you visited any of the sites above you’ll see that each of these examples had the candidate perform a series of steps. If the candidate is not sure of what to do, they could ask me for clarification or simply Google for the answer. When the candidate is done they’re to submit a pull request to me so I can review it.

This challenge is genius because it answers so many questions about the candidate. Namely, can they code and get something done on time. It also gives me the chance to review the code to see if it is a complete hack or if its something that is quite good. What I’ve found is that on average over 60% of candidates will never finish the problem – they simply don’t know how. Of the 40% that finish only about 20% actual do it correctly and usually at that point you’re left with either 1 or 2 candidates that are looking good. If you have more to choose from then thats a good problem to have!

It is also important to note that ODesk requires that the candidates install monitoring software that takes periodic screen shots of the work they’re doing so that you can review the work. This allows me to view their progress and semi-evaluate their process of solving the problem.

A common concern when letting someone do this task is: What happens if they cheat? What if they call their friend or have someone else do it for them? That is a possibility, but I see this as a real world problem. Sometimes programmers don’t know how to solve a problem so they have to call for help. They call friends, IM them, look stuff up on the internet or just find an open source project that does what they want and they take the code and modify it. Thats the world programmers live in. If I do that at an office or at home, who cares. If you’re truly a hack I’ll know fairly quickly. If you do get hired, I’m going to sniff out a problem very quickly and at that point you’ll need to make the decision to keep the person or not. If you’re a business person who can’t sniff this out, you may notice other things slipping such as timeframes, budgets etc. Any red flags are exactly that, red flags – so evaluate each one with caution.

If the candidate fails to complete the programming challenge, thats an immediate red flag. The task should be easy enough for an experienced programmer to finish in an hour. If the candidate cannot finish within the time allotted either the programming task is way too hard or the candidate is not suited for the position. If you’r using my Git repos from the links above (or something very similar) then this should not be the case. At this point I advise to stop the interview process for those who cannot complete the task and move onto the next candidate.

For those that do solve the problem correctly I like to review the code and if everything is satisfactory I’ll put them on the list of potential hires.

If you’re someone who cannot code, but you want to follow this process I highly suggest you find a friend, friend of the family, or coworker who is a programmer and ask them to help you. Offer to pay for 2 hours of their time so you can get their opinion of the candidates. Explain to them the process you’re going through so they know what their role is – a code reviewer. If you use this approach you will want to give the code reviewer instructions to give you a scale of “No/Maybe/Yes” for each code review. No means No – do not hire. Maybe means – its ok, not perfect, but workable. Yes means Yes, hire this person, this is good work. I include maybe because sometimes code looks good, but its not perfect. Thats OK. Sometimes you need to get out an MVP of a product out the door ASAP and it does not have to be perfect, it just has to work. The “Maybe” candidates are suitable for this because you’re testing a market/etc and you’re trying to get things done as fast as possible. However, you want to aim for YES candidates at all times if possible.

If you don’t find anyone who can solve the problem, repost the job posting to get some new leads. I’ve had to post a job a couple of times to get the correct person for the job. Yes, this costs more money, but it is much cheaper to pay this up front than months down the road.

Step 4: Keep the Top Candidate(s)

Goal: To hire the proper person/people for the position

In the previous step you identified who you can hire through code review and through initial filtering based upon communication through the application process. At this point you should be ready to hire your contractor. You may have had 2-3 people make it this far.

If you’re hiring for one position, hire the top candidate and let the other two know that you decided to go with someone else but you may be in contact with them shortly because you were impressed and you may need to grow your team. This leaves the door open for future conversation with those candidates. If you need to hire two people, hire the top two. You get the point here.

If you don’t end up with enough contractors, then repost the position.

Wrap Up

Hiring a contractor can be a challenging task, but if you follow the steps outlined above you’ll find that you can lock in a good outsourced developer and eliminate the cruft in a few easy steps. I hope this helps you land a great outsourced developer.

* I’d love to take full credit for creating the programming challenge on my own, but I cannot. The implementation of this challenge was inspired by a company by the name of Integrum (who no longer does programming) for this method of programming challenge.

Android Dev Digest

Get the best Android Developer posts delivered weekly to your inbox.

Don't worry, I wont spam (I hate that stuff too).

  • kstop

    If this process was in widespread use, I’m guessing there would be a lot of shady devs optimizing their resumes to win 1-hour contracts ^_^

  • Critic

    This article itself does not even earn a 5/5 English Level rating.

  • http://blog.donnfelker.com/ Donn Felker

    The 1 hour contract is to ensure the person I’m about to hire does the quality work that they say they can do. If they complete that task successfully and I feel the code is sufficient then they will be hired. This would be on going work. Simply optimizing your resume for 1 hour contract wouldn’t guarantee a hire because the goal is to evaluate real work done by the candidate.

  • pursera

    This is a great article on hiring!

  • Simon

    I really appreciate you writing this. Thanks.

  • Padmarag

    link to github api is wrong. it should be developer.github.com not deveop.github.com

  • http://blog.donnfelker.com/ Donn Felker

    Thanks for the catch. Fixed.

  • http://www.theroadtosiliconvalley.com/ Ernest Semerda

    Nice insights. Thank you for sharing.

    This is simple. If you are hiring an engineer you should be one too. This way you can hire faster and fire faster. No bullshit.

    Outsourcing work overseas is 5x cheaper than local try-and-see-how-it-goes. Not a long term solution but a short term one to get stuff done. Hire fast, fire fast. So if you make a mistake move fast to get another one on the job. You wont bleed as much money from a mistake of outsourcing to the wrong candidate.

    I have been through this a number of times over the years and documented my learnings on outsourcing pros & cons here: http://www.theroadtosiliconvalley.com/technology/outsourcing-work-overseas/

  • Lou Prado

    Thanks for this article. I’ll follow your process this weekend to hire an iOS developer to help finish my code. BTW, this post made the front page of HN! Congratulations!

  • http://blog.donnfelker.com/ Donn Felker

    Thanks Lou! I hope all is well with you! Great to hear from you. :)

  • http://openbsd.org/ mmontoya

    This is a good article but I think you are not considering an important aspect and is the previous projects and the “footprint” of the candidates.

    If I got an answer of some candidate and he (or she) gives me a github account and I can see he is the author of things like:

    https://github.com/cloudinary/cloudinary_gem/blob/master/lib/cloudinary/api.rb

    And I found he es member of Rails-Croatia group and he answered several questions to the newbies and he has a blog explaining the technical differences between Puma and Unicorn the “one hour programming task” will be useful only to know his communication skills.

  • Mike

    “tram” should be corrected to “team”

  • b

    “If you’re hiring for one position, hire the top candidate and let the other two know that you decided to go with someone else but you may be in contact with them shortly because you were impressed and you may need to grow your tram.”. Grow your tram. :)

  • Jeff Potter

    Great article. I learned a lot. I am thinking about taking your Android developer challenge. I think it shows a lot about their comfort with understanding instructions, knowledge about GIT, and quality of their code. Thanks!

  • http://blog.donnfelker.com/ Donn Felker

    Fixed. Thanks!

  • http://blog.donnfelker.com/ Donn Felker

    Just fixed it. Thank you.

  • Freelancer

    I am a very decent developer and sometimes work on oDesk. If you try to make me use the piece of shit oDesk screen tracker, you will lose me immediately.

    The bottom line is, you either trust me enough to judge my work by the results or you don’t. If you don’t, making me install spyware software is just going to alienate me further. You’re the remote equivalent of the boss who drops by my desk every 15 mins that everybody hates.

  • Dieter Engel

    I think you and the person who wrote this article don’t know the difference between git and github. Also, expecting someone to learn an arbitrary API (or to already know the github API) is silly.

  • Dieter Engel

    No decent iOS developer will put up with this kind of nonsense. I say this as an accomplished iOS developer. But then, decent iOS developers charge $155 an hour, not $20 an hour.

  • Dieter Engel

    Except that outsourcing overseas takes 7X longer, and thus is not a savings. You can get people who will follow these tasks, and can make a half assed android app that will show github issues.

    But if you need to do an actual piece of software that is not trivial, you will find that overseas people simply cannot (or will not- I think they tend to work for dozens of companies at once) spend the time to do it.

  • Dieter Engel

    This is a process that will find you coders, e.g.. PSD to HTML types. You will not find any “Developers” or “programmers” or “engineers” with this methodology.

    First off, you’re not testing programming ability here, you’re testing to see if someone has already written something that uses the github API. (EG: if you think writing an android app that uses the github api is only an hours work you are underestimating the amount of work.)

    Secondly, engineers/programmers/developers are people who create original work that solve problems that are particular to the needs of the product they are creating. You’re not asking them to do this, in fact you’re not asking for any programming, just coding.

    I guess on odesk coders are what you will likely find anyway… but let me assure you, you’re not going to be finding any good programmers with this method.

  • Dieter Engel

    Ah, I get it, you’re not a programmer. That’s the problem. Non-programmers telling other non-programmers how to hire programmers.

    Blind leading blind.

  • http://blog.donnfelker.com/ Donn Felker

    I mainly use the screen sharing software during the interview and review process. Once I trust the developer to do their work (usually happens quite quickly) I never check the log again. Most of my developers stop using the tool and enter their time manually after that. You’re right, its all about trust. I simply don’t have time to be a micromanager, nor do I want to. Again, I use it as a review tool to help make decisions around who to hire.

  • http://randyburden.com/ Randy Burden

    Good tips! I love how your node and android programming challenges are essentially screenscraper apps… that was actually my first paid gig!

  • Dennis L

    “Something interesting happens at this point in the process. An average
    of 50% of the candidates never continue past this step. The reason is
    not exactly known but I suspect it is because they simply do not know
    how to perform the task that is given to them.”

    Or they don’t like being insulted. I’ve written software for Apple, Sony, and Hitachi, including for PlayStation @ Sony. I think I’m good enough for you without your little “test”, which was probably created by a non-programmer. Also 1 hour isn’t enough time to do much of anything program wise (beyond “hello world”) and it’s sure not a good gauge of a person’s skills.

    Real talent doesn’t like being insulted.

    Next time I need brain surgery I’ll ask the surgeon to take a test for me, or to cut up some mice so I can see how he does.

    Yeah, try that with good surgeons and see how many will agree to operate on you.

    The only real way to judge developers effectively is to see their track record and what they’ve actually delivered, not quiz them like they’re on a game show.

    Great developers are seriously talented people. Treat good talent that way and they’ll think YOU’RE the joke and walk instantly. It’s not because they can’t do the task.

    Were you instrumental to the success of PlayStation at Sony? What’s that, you weren’t? Well I was. Then just who do you think you are quizzing me?

    Maybe I should be the one giving YOU the quiz to see if you even know what you’re doing.

  • Dennis L

    Have you ever written code? What products have you written? If not how are you qualified to eval potential developers?

  • Bad Spelling

    “A lot of contractors on ODesk will say their 4/5 but when you converse with them via email, IM or Skype/Google Hangout you’ll realize they’re actually more like a 2 or a 3.”

    If you’re going to say you want a 5/5 in English, you likely shouldn’t misspell “they’re”. Very amateur…

  • Jon Watte

    I came up with the same process for a task I wanted outsourced. I had 0 developers acceptably solve the initial project. Maybe I was just unlucky? Or maybe it’s because the domain is hard core Linux C++, which might have a smaller candidate pool?

  • Impressed by your skills

    1 hour to write “hello world” ? Damn, you *are* good.

  • Savvas Pavlidis

    The point here is that he wants someone on low cost. Real good programmers with actual good record with “heavy” names in industry would require far more than the maximum limit of 25$/h that Donn Felker said. Thus his target is good programmers with no previous heavy portfolio, that would work within low budget, and if I take into account the first rule, the good knowledge of English, I assume that most candidates are far away from USA with other mother tongue. So, his tests is not an insulting procedure, if you take into account which type of candidates actually he aims.

  • hanspeide

    Uhm…:

    they’re

    contraction of
    1. they are

  • Mădălin Constantin Putină

    Perhaps he’s referring to people who’d consider aiming for as many 1 hour contracts as possible, without actually putting any effort into serious attempts at being hired.

    Imagine someone with a CV so convincing that numerous employers offer this person a 1 hour contract as a test. This person would then proceed to cash in on this 1 hour fee, without actually producing any useful work; they would only need to optimize for as many 1 hour contract chances as possible.

    People begin to suspect foul play and write about this person? No problem, just ditch the account, create another fake online persona and proceed to gather a new round of 1 hour contracts.

    Soon, as the number of fake candidates increases, you’ll find that it’s much more profitable in the long run to hire based on credentials and on a past history of successful projects, even if you’d pay more in hourly fees.

  • http://www.anupshinde.com/ Anup Shinde

    So you pay only $25/hr to a person who solves that challenge in 1 hour? (I checked NodeJS one). What is your success rate of hiring such a person at that low rate?

    Nice article though!

  • tbsprs

    Please check the usage of “its” in both articles. It should be “it’s” or “it is” in many cases I suppose.