Where have I been?

I haven’t posted much here lately, but I’ve posted some stuff about offshore development at my other blog: http://cayugasoft.com/blog/.  Originally I had intended to focus the Camiapp blog on logistics and supply-chain technology, and perhaps the cloud, but I had offshore software development on my brain, due to some projects I was working on.  So I think I’ll return to logistics/supply-chain/cloud here in not too long, but if you’re more interested in agile offshore development, check out CayugaSoft.


Four Ways to Do Offshore Development

Some people think that offshoring software development = outsourcing software development.   That’s not the case.  You have multiple options.  Let’s talk about the main options and how they may or not make sense for you:

1)      Outsourcing (Project Based) – If you may have peaks in valleys in your workload (e.g., you have a project that requires a boost in resources for a period of time) it doesn’t make sense to hire full-time staff to address a period of peak demand.  In this model, you bid out your project to one or more software development outsourcing companies, and pick a winner.  This is what most people think you are talking about when it comes to doing software development offshore.  It makes a lot of sense to outsource if you don’t feel that software development is a particular competence you need in house in your business.  Or you may think it’s worth investing in your own staff, onshore, but you want to access some lower cost resources offshore from time to time.  A downside of this model is that you don’t really get any leverage from the investment you make in training the developers on your business and your systems.  The developers scatter to the winds when the project is done.   So, in most cases, this isn’t a model I favor.  Experience shows that it doesn’t work all that well—you often get a product of questionable quality.  On the other hand, if you need some very specific skills for a project—experience with integration to a certain ERP system, or porting your application to a certain mobile device with which your developers have no experience, then this approach could make some good sense.  Perhaps you don’t need that skill in-house for the long-run, and hiring an expert for the duration of your project makes a lot of sense.

2)      Outsourcing (Dedicated Team) – This is like #1, except in this case you recognize that you have a constant need for software development resources to develop subsequent iterations of your product.  This is really a norm for successful software development companies.  Perhaps you are using Agile sprints, where you release new functionality periodically—say every 3-4 weeks.  And when a sprint is done, you do another one, based on a backlog of requirements that you are working off.   So there’s always some new development to be done, as long as new customers and revenue keep coming in.  So instead of bidding the project out, as in #1, you contract for a period of, say, 2-3 years with a company that provides you with a dedicated team of developers.  You invest in ramping up the team to the point where they understand your product, your development process and your business, and the team gets progressively more productive as their knowledge and understanding grows.  A lot of offshore development companies will work with option #1 if they have to, but they prefer option #2.  It means a constant revenue stream for them, and in the end they’ll be delivering a better service to the customer as the team’s capabilities grow.

3)      Captive – A “captive” offshore development shop is one that works for only one onshore customer.  In other words, the onshore company creates a legal entity in the offshore country and builds the software development center the hard way: Hire a lawyer, hire an office manager, rent office space, hire team leads and hire developers, testers, contract for a cleaning service and healthcare insurance, etc.  Doing all of these things means that this approach is going to take a lot longer to start up than the others.  And there is some risk it just won’t work (e.g., you have no track record in the country and you have trouble hiring good staff).  But over the long run it could cost less than outsourcing, and you have the opportunity to create a great culture and, hopefully, benefit from that with low turnover and staff that will go the extra mile to deliver for you.  But the savings will only come when you reach a certain scale, so you can spread all of your overhead and up-front costs and hassles over a large number of developers.  What scale do you need?  In my experience you need at least 25 developers offshore to make it worthwhile, and maybe more (and certainly this would depend on the offshore location you choose).  So it’s worth considering a captive operation at that scale, but if you’re not sure you need that many, it’s probably not worth it.  If you decide to build your own captive operation, what you won’t get is the benefit of the offshore development company’s expertise.  Some offshore development companies have made some good investments in processes and tools.  For example, if you visit an offshore development firm, ask for a demo of their automated test tools and see what they can show you.  Are their capabilities better or worse than your own?  In a lot of cases you’ll find they are better.  If you are building your own shop, you can’t benefit from those capabilities—you’ll need to invest on your own.    But if you want to get the lowest long-run costs and you can tolerate a slow start-up time, a captive shop could be an option for you.

4)      Build, Operate, Transfer (BOT) – BOT is like #2, except the development shop agrees to transfer the team to you after some period of time.  So, for instance, at the end of a three-year contract, you might have the option to buy out the team from the development shop, and the employees become yours.  Some development shops will agree to do this, and some won’t.  And the ones that do are often hoping you won’t bother to buy the team out and they’ll get another few years of revenue from you.   The advantage of this approach vs. creating a captive site from scratch, is that you should be able to fire it up much quicker, since you can take advantage of the offshore development company’s pre-existing recruiting capability, office, development processes and tools.  Of course you’ll pay for that, but if you strongly feel that the staff should be yours, and not the offshore development company’s, and you are in a hurry to get started and want to reduce your risk, BOT might be a good option for you.

All of these are viable options and the one to choose depends on your needs.  The only one I don’t like much is project-based outsourcing (#1) since you don’t benefit from the investment you make in training the developers on your business and systems.  If you have a one-off project, or a relatively simple project, or one that requires some very specific skills that you don’t have in house, that might be OK, but otherwise take a look at options 2,3, and 4.

Tagged , ,

The List: Best Countries for Offshore Software Development (Part V – Last in the Series)

Today I’m going to wrap up this series on the “best” countries for offshore software development by giving you my list of top countries.  Yep, I’ve given you a lot of detail on how to arrive at your own short list of countries, but today I’m finally going to just give you my list!

I started this series by looking at Gartner’s top 30 list of countries for offshore services.  We looked at the criteria Gartner used to come up with the list, critiqued those criteria, and modified the criteria to ensure they were useful for evaluating countries for software development, as opposed to more generalized offshore services, which might include help desk support, or Business Process Outsourcing (BPO) tasks.  OK, so if we apply my criteria, what is the list of countries we might come up with—the “best” ones for software development?  We could do this at a granular level, by assigning each country scores on a scale from 1 to 5, as Gartner does, and weighting these factors to come up with total scores for each country.  I think it’s worth doing as part of your analysis since it forces you to think through all of the criteria, and their importance to you.  For our purposes today, though, it’s simpler and quicker to start with Gartner’s list, and just knock off several countries that don’t fit my criteria: cost, labor pool, educational system, language and cultural compatibility, proximity to your business, political and economic environment and global and legal maturity (I combined a couple of Gartner’s criteria, here).

Most of the countries on Gartner’s list just don’t have a critical mass of strong development talent.  I’m sure you still could make a go of it in many of these places (e.g., I know someone who does a fair amount of work in Sri Lanka, so I know for a fact you can make it work there, but I’m don’t think it quite makes my list of “top” countries).  So I’m going to knock out Costa Rica, Panama, Peru, Bangladesh, Indonesia, Malaysia, Philippines, Sri Lanka, Thailand, Vietnam, Czech Republic, Egypt, Hungary, Mauritius, Morocco, Poland, Slovakia, South Africa, and Turkey (for more detail on the Eastern European countries—you might wonder why I knocked out Poland, for instance–look at this analysis to see why some countries don’t quite make my list).  Chile is a bit small compared to the others (the population is only about 16 million), and it seems that the talent pool there is a bit thin (if you talk to offshore development firms in Chile, they tend to be smaller than in other major offshoring destinations, and sometimes Chilean firms might outsource some of your work to, say, an Argentine firm because they tend to have more resources than a Chilean firm).  However, I think you could be successful in Chile if you’re not trying to build a very large team.  So I’ll leave Chile on the list, especially since it has some other good qualities such as stability, a good political and legal environment, and a good cultural fit with North American or Western European businesses.

Some of the countries are getting expensive—Russia is an example—so I’m going to knock them off the list too.  And the business environment in Russia is discouraging to most Western business-people, so I’ll also knock them off the list for that reason–most Western businesses just don’t want to invest there unless they are involved with oil and gas.  Brazil is also getting more expensive.  Sao Paulo recently made a top ten list of most expensive cities to do business.  They’ve found offshore oil, and as one of the BRIC countries, large companies feel they need to have a presence in Brazil, so those factors are heating up the market for the rest of us.  I’m tempted to leave Brazil off the list for this reason, but if cost is not your primary driver, or you have other good reasons to be in Brazil (e.g., you are targeting the Brazilian market with your products) maybe you should consider it.

I feel that Mexico should be an interesting alternative for US and Canadian firms, but it’s hard to make a case that it’s among the best places in the world to do business these days, due to the drumbeat of crime stories we hear from Mexico each week.  So I’m knocking Mexico off the list. Colombia is an interesting one.  I haven’t been there, myself, but I keep hearing good things about the talent pool there.  And there is a good-sized population there (46 million).  I just spoke to an old friend of mine who lives in Colombia, and while he likes some aspects of the place, he didn’t rave about the business culture and the ease of doing business there.  Apparently Colombia is much safer than it used to be, but like Mexico, it might be hard to make a case to your board of directors that you’ve found the perfect place for a foreign investment, and it’s called…. Colombia?!?!?  So I’m going to leave it off my list, but it might be worth considering if you are aggressive and want to get in early on a place that’s on the upswing, or if you have some special insight into the country (which I don’t have).

So if we knock those countries off of Gartner’s list, our list looks something like this.

  • Americas: Argentina, Brazil, Chile, Colombia, Costa Rica, Mexico, Panama and Peru.
  • Asia/Pacific: Bangladesh, China, India, Indonesia, Malaysia, the Philippines, Sri Lanka, Thailand and Vietnam.
  • Europe, the Middle East and Africa (EMEA): Bulgaria, the Czech Republic, Egypt, Hungary, Mauritius, Morocco, Poland, Romania, Russia, Slovakia, South Africa, Turkey and Ukraine.

In Asia, I’m left with the two giants, India and China.  The talent in these countries is undeniable, and they scale well—you can find people in quantity, and the costs are reasonable.  The language skills in China are better than they used to be, but based on my visit there a little over a year ago I’d still rate them as “poor”.  You might arguably rank India and China below the countries in Eastern Europe or South America in terms of cultural compatibility (it’s a subjective criteria, so you can judge for yourself) and proximity to your business, and these days there is a lot of interest in “nearshoring”—offshoring a bit closer to home or in time-zones that are closer to your own in order to improve collaboration.  But India is still the global leader in offshoring and will continue to be for some time in the future.

I’m going to add Belarus to the list.  Gartner doesn’t consider them one of their top 30 countries for offshore services, but if we’re talking about software development, in particular, I think it makes the list.  As I showed in my prior analysis of Eastern European countries, it’s a place with a good population of strong developers and a relatively small internal IT market, so they are very export oriented when it comes to software development.  On the other hand, the politics there are a real worry and it’s not a place to go if you don’t have some risk tolerance.

So the revised list might look something like this:

  • Americas: Argentina, Brazil, Chile
  • Asia/Pacific: China, India
  • Europe, the Middle East and Africa (EMEA): Belarus, Romania, Ukraine

These countries fit my criteria—they have populations that are large enough to have a “deep” talent pool (with the possible exception of Chile), with education systems that are good enough to produce software developers in sufficient quantities, the business risks are manageable (although they vary pretty widely), and the costs are reasonable vs. the US and Canada, and Western Europe.  If you’re looking for a “wild card”, you might also consider Colombia, or perhaps Bulgaria, where there is a small population, but a decent number of developers relative to that population.  I’m probably going to get some complaints from some of you that I left your favorite country off the list.  By no means am I saying that you can’t do development in countries such Vietnam or the Philippines, particularly if you only need a small team (e.g., 2-12 people).  But if you want to do software development at scale (i.e., 12-100 developers), I think you’d be better off in the countries that I’ve listed.


Best Countries for Offshore Software Development (Part IV)

In my recent posts, I took a look at some of Gartner’s criteria for determining the “best” countries for offshore IT services, and told you that Gartner’s framework is perhaps better used for Business Process Offshoring (BPO), and more generalized IT services (e.g., customer support and help desk), rather than software development.  So some of the countries on Gartner’s list of top 30 offshoring destinations would surprise you if you were looking for a list of great software development nations (Egypt?  Panama?  Slovakia?).  But if we’re looking to narrow down our list of top software development offshoring countries, we can still get something useful out of the Gartner criteria if we’re a bit critical in how we apply it, and we make a few adjustments.  So let’s talk about the criteria that I think are most important for evaluating offshore software development destinations, and then in my final post in this series, I’ll give you my own “top countries” list.  Here are my criteria for evaluating a location for software development:

Cost – For obvious reasons.

Labor Pool – I like to see a good critical mass of skilled developers in a country and, subsequently, in a city within that country.  If you can’t scale your team beyond a few people, why would go to that country?  So while Gartner has countries like Cost Rica (population 4.3 million), and Slovakia (population 5.4 million) on their list, I don’t have much confidence that you could find large numbers of good developers there.  These are pretty small countries with small populations, and there’s some risk you’ll hit bottom if you need to hire more than a few software developers.  I prefer countries that have a larger population.  Places like India and China have huge populations and so it’s just a law of numbers that there will be some well-qualified people in those countries.  But you don’t necessarily need to go somewhere quite that large.  Countries like Ukraine (population 46 million) and Argentina (40 million) are big enough that you should be able to find the skill-sets you need, for the most part.

Educational System – This is an important criteria, and is related to the labor pool—is the country producing well-trained developers each year, in good quantities?  In most of the countries on the Gartner list, the universities are just not top tier.  [Check out a list of the top universities in the world.  The top universities generally aren’t located in Gartner’s top 30 offshoring countries.  You’ve heard so much about the IIT schools?  They are among the better schools in the developing world, but they don’t rank all that well on a global basis.  Ukrainian or Russian schools?  Fuhgeddaboudit.]  But in the case of software development, the universities in many of the countries on Gartner’s list are good enough, and given the highly motivated students they have, the graduates they produce are very strong.  The good thing about software development vs., say, particle physics, is that you don’t need outstanding laboratories and research facilities.  I had a chance to visit the Polytechnic in Odessa a couple months ago.  It doesn’t look like an Ivy league institution, to say the least.  Luckily, running a computer science program at a high standard just isn’t that expensive these days, so you really don’t need fancy facilities for this, assuming your focus is on training smart, motivated practitioners, and not on conducting world-class research.  So while the universities in the top offshoring countries don’t rank very well on a global basis, they can still produce very good development talent.

Language and Cultural Compatibility – The importance of this factor depends on you.  As in my last post, I don’t think these factors are so important when you are talking about enterprise software business domains that have a long learning curve (but not everyone agrees with me on this one).  In that case, you need to invest in training your team on your business domain, and their “culture” won’t give them much of a head start.  But for consumer-oriented products, I’d consider these factors to be more important.  So, for example, while I’ve done a lot of enterprise development in Ukraine, I might look at Argentina seriously if I was looking to find additional resources for a consumer-oriented product whose market is in North America.  Romanians like to make the case that their culture and language skills are quite compatible with North America or Western Europe.  Because the Romanian language is Latin-based, and uses roman characters just like in English, this gives them an edge in learning other Latin-derived languages.  One more factor to consider is turnover in your workforce.  I don’t know whether to consider it “cultural” or not–perhaps it’s more related to how hot the labor market is in a country–but let’s consider it to be part of the “business culture” in a country.  Some countries have had a history of very high turnover rates.  In North America, we don’t leave our employer every few months (assuming the work is rewarding, we’re being treated fairly, and the pay is reasonable) .  We’d like to see a similar dynamic when we offshore software development to another country.  If that’s not the business culture in a country, Americans or Western Europeans are going to be disappointed with the results of offshoring.  Costs will be higher than expected and productivity will be lower, since if your developers keep turning over you’re going to have to continually invest in training new ones.

Proximity to your Business – This isn’t broken out in the Gartner criteria, although they mention it under “cultural compatibility”.  I’ll break it out and discuss it separately, since today a lot of people like to talk about “nearshoring”–being close to your developers in terms of distance or time zones.  So South America has some attraction if you are in the USA, since the time zones are almost the same as in the US, while Eastern Europe is attractive if you are in Western Europe.  At a minimum, you’d like to have some overlap each day so that you can talk to your developers, and so that they can participate in Scrum meetings, etc.  I’ve found you can still make Eastern Europe work from North America since you can still get about a half-day’s overlap in your business hours, but I had a chance to work on a some projects in China and Japan a couple years ago, and conference calls with North America were always an inconvenience to one party or the other, and this was a real inhibitor to resolving issues quickly.   India has a lot going for it, but the time difference with North America is a challenge when it comes to collaborating with North American developers and product managers.

Political and Economic Environment, Global and Legal Maturity – The relative importance of this one really depends on you.  As I’ve argued before, there’s a potential “opportunity” to working in a less stable country, since the costs may be lower.  Think Ukraine or Belarus vs. Poland, for instance.  But if you can’t tolerate tax systems that are a moving target, or some political and economic risk, then by all means, pay a bit more and go somewhere more stable.  For example, Chile is a place that impressed me when I visited there a few months ago.  If you are considering making a direct investment in IT in Chile, the government will help host your visit, set up meetings with similar companies who can share their experiences operating in Chile, and they’ll give you some interesting tax incentives if you want to open your own “captive” subsidiary.  They’ve really got their act together, and while it might cost you more in Chile vs. some other places, it might be worth it.  (The only potential issue I see in Chile seems to be the depth of the population and talent pool there–it’s a small country with only about 16 million people, but if you are setting up a small or medium-sized team it would probably work out fine.)  If you are considering going with a less “mature” country, it might make sense to go with a services provider in that country vs. opening your own captive development center, since the provider will likely know how to operate safely in that country, and they take on most of the risks.  Either that, or you’d better hire some people who really know what they are doing in that country.  On the other hand, you could go with a provider in a more stable country, but opening your own subsidiary might be quite feasible there, if you are sure you want to make an investment for the long haul and you aren’t just dealing with a finite-duration project.

In summary, focus on countries with a decent-sized (and skilled) labor pool, a good educational system (particularly when it comes to producing large quantities of technically skilled people), reasonable cost, and good proximity to your home country.  If cultural fit is important to you, that might affect your preferred countries as well.  You should also think about the relative political and legal stability of a country and consider how much risk you can take, since for some extra money you can probably reduce risk, but on the other hand, if you can tolerate a little risk you might save some money.  You can use Gartner’s ratings for the above criteria, but I think it’s better to adjust these ratings with your own research and score each country based on these criteria.  You can weight each factor as you see fit.  As an example, if you look here, you can see how I came up with the “top” locations in Eastern Europe, considering some (but not all) of these criteria–I was mainly just considering cost and having a “critical mass” of development talent, although I threw out Russia due to political risk.  Next, you can create a short list of countries that seem to have a good fit for you.  Once you’ve done that, you can consider specific cities within those countries, do more research about service providers in those locations, and consider the pros and cons of opening a captive center in one of those locations vs. using a service provider.

In my next post I’ll wrap up this series by posting my own top list of countries for offshoring software development.  Stay tuned to see if your favorite makes the list.


Best Countries for Offshore Software Development (Part III) – Cultural Compatibility

This is Part III of my post on a how you might think about determining where to locate your offshore software developers.

I started in Part I with the Gartner list of top 30 IT offshoring destinations, and have been critiquing the applicability of Gartner’s ranking criteria for software development, as opposed to more general IT services tasks.  Gartner has a criterion they call “cultural compatibility” that they use in rating the various countries.  It’s one of ten criteria they use, but I’m wondering about its applicability in a “flat world”.  Gartner doesn’t define cultural compatibility in much detail, but they say it involves “adaptability to multicultural approaches to support an international business environment”, and they also consider the time zone of a country, ease of travel to a country, and its proximity to the major home countries that purchase offshore development services.

Some of the potential attraction of South American countries (at least to a North American company contemplating doing some software development work offshore) is their cultural compatibility with the US (that, and the closeness in time zones to the US).  So what about it?  Should you consider cultural compatibility?  I think so, but it might depend on the software products you are developing.  I had a chance to visit South America recently, and one Argentine software development executive noted that “we watch the same TV shows”.  I don’t think watching CSI is going to make a developer more productive (but CSI Buenos Aires might be a show I’d watch!), but of course that wasn’t his point.  What he meant was that, with some common cultural understandings, Argentines could communicate well with Americans and more quickly gain a common understanding of what a project was about or, in Agile terms, what a user story meant.  That’s worth something.  What I’ve noticed is that when the project at hand is consumer-related, an Argentine firm might arguably have a better understanding of US consumer society than developers who live in farther-off time zones.   I met with the head of an Argentine software development company last week and it struck me that his client list is very strong in advertising agencies, social media, and other consumer-oriented work, although they do work on business applications as well.  In my case, I’ve spent most of my career developing business applications where developers learned the domain over a period of years, by working with their US counterparts.  They didn’t come with an innate understanding of the business processes related to that domain (logistics) any more than a US developer does.  So in cases such as this—business applications—I’m not sure culture is as important as training your developers on your domain, and finding a way to have a low turnover so you don’t fritter away the investment you make in training your people, whether they are your own staff or those of an offshore development company.

Let’s talk about work habits, which might be one aspect of “cultural compatibility”.  Something I’ve heard more than once from US senior managers is, “…those Eastern Europeans are still shaking off communism, so how do you get them to work hard and show up every day.”  Look, the Berlin wall fell in 1989.  That’s almost 23 years ago.  In Eastern Europe, your average developer is probably going to be about 28 years old.  So the residual effect of communism on the average developer’s work habits is going to be negligible.  Now, it’s true that bureaucracy continues to smother some of the less economically advanced economies in Eastern Europe (and for that matter, most major offshore destinations, such as India).   As an example, I pointed out in my last post how difficult it is to pay your taxes in some of these countries.  So bureaucracy is something you are going to need to deal with if you operate in those countries.  But in my experience it has no bearing on your developer’s commitment to their jobs and their companies.  If you can motivate them with interesting projects they’ll show up for work, try hard to understand your business and work hard to deliver (at least they will if you’ve recruited well).

In the long-term, I think it’s clear that the world is becoming flatter, and cultural factors may be less important when considering where to offshore, although it’s still a factor now.  If you travel in these countries, you’ll see that everyone is on smartphones, using LinkedIn or Facebook (or their own local variants of these networks), so people are slowly growing more similar, not more different (at least when considering the major offshore development destinations, as opposed to, say, Saudi Arabia).  Many years ago, I worked on a consulting project with IBM that involved some work with an R&D lab in France.  Over and over, cultural differences came up.  I think both the French and the US teams respected each other, but there were differences in how they approached development projects and in the processes they used to make decisions, and resolve issues.  Today, methodologies like Agile are providing people with a common language for developing software.  It’s a global language, and with more and more projects being developed across multiple sites and countries, people are developing a common understanding of that language.  An Agile “Product Owner” is a Product Owner, regardless of what country they are located.  Sure, people will bring their own cultural backgrounds to roles such as this, but I see the importance of these differences declining, as least as it pertains to software development.  Twenty years ago, the idea of developing products using mixed onshore-offshore teams was daunting.  Now, it’s commonplace.

Tagged ,

Best Countries for Offshore Software Development (Part II)

Today, I’m going to continue from my last post about Gartner’s criteria for determining the “best” offshoring destinations, and consider how relevant these criteria are for offshore software development, as opposed to Business Process Outsourcing (BPO) or IT services not related to software development (e.g., help desk and other IT support tasks, systems monitoring, etc.).  Here are the variables Gartner uses to rate the various countries:

  • Language
  • Government Support
  • Labor Pool
  • Infrastructure
  • Educational System
  • Cost
  • Political and Economic Environment
  • Cultural Compatibility
  • Global and Legal Maturity
  • Data and Intellectual Property Security and Privacy

Gartner gives each country a score on these dimensions, ranging from “poor” to “excellent”.  Gartner has detailed reports on all 30 countries on their list, and if you are a Gartner subscriber they’ll let you see those reports.  If you aren’t, you can probably search around the Internet and find some of the data, but I’ll leave that as an “exercise for the reader”.  Let’s talk about some of Gartner’s criteria in more detail.  Some of these are clearly going to factor into your decision (e.g., cost).  But what about some of the others?

Language–Language skills in a country are important, but I would argue that this is not as important as Gartner seems to think, at least when it comes to software development.  Most software development shops need good English skills from at least some of the developers (e.g., team leaders).  Some people insist that all their developers speak strong English.  I think that’s a bit extreme, and risks excluding some good developers from your applicant pool, although I think it’s great when I see mandatory English lessons when I visit an offshore software development service provider.  Gartner seems to rate the countries based on the general language skills endemic to the population.  So countries like India and the Philippines do well because English is used as a language of instruction in schools.  If you’re doing BPO work in a country, and need to hire 500 call center workers, then I agree–you need good language skills in the general population if you’re going to find that many call center workers.  But that’s less critical if you want to hire 30 or 40 software developers.  Then all that matters is your ability to attract 30-40 good people, at least some of whom speak good English.  The level of English in the general population doesn’t really matter much, in that case.  In addition, most developers can at least read English, even if their spoken English may be weak, since they’ve had to read technical information in English.  So, in summary, the language skills available in a country are important, but not as important for software development as for BPO.

Infrastructure – Today, most countries that are worthy of serious consideration have infrastructure that’s good enough to support software development activities.  So I think this criterion is of diminishing importance, and frankly some of Gartner’s ratings here are a bit confusing.  Ukraine, for example, gets a “Poor” infrastructure rating from Gartner.  I can tell you the roads in the major cities in Ukraine are perfectly fine.  In fact they are better than in parts of New York City.  I hear the roads are poor in the Ukraine countryside, but so what?  Your software developers are going to be located in a major city and you are going to fly in and out of that city to meet with them.  The electric power in Ukraine is a bit worse than in the US, but it’s not terrible, and the Internet is decent.  That’s about all you need for software development.   Meanwhile, Gartner gives India a “Good” rating for infrastructure.   It’s been a few years since I’ve been in India, so maybe things have improved, however I don’t meet a lot of people who’ve been to India who would rate the infrastructure there as “Good”.  Although it’s certainly “Good Enough”.  Apparently they did a nice job with the subway in Delhi.  And I hear Starbucks is finally opening up in India, which is an important part of the infrastructure, according to me.  (In 1999 I can remember visiting multiple Starbucks in China, while all these years there have been zero Starbucks in India, so this is actually an interesting barometer of something–India’s openness and attractiveness to foreign investment, vs. China’s, I suppose).  So anyway, the infrastructure in a country might have been more important ten years ago, but I don’t find it to be that important today when considering where to locate software development.  Again, if you’re putting up a 500 seat call center maybe it’s a different story.

Political and Economic Environment, Global and Legal Maturity and IP Security – In my last post I joked about Gartner considering Egypt to be a viable offshoring destination, just before they had a revolution, during which the autocratic government there shut down the Internet–possibly the first time any country has ever been taken off the net in this way.  You don’t want to be in a place that’s one strep from chaos.  If a country is very unstable your developers will have an incentive to find ways to leave the country.  And you may have some banking system disruptions that make it difficult to pay them, which happened to me in Ukraine in the financial crisis in 2008.  We sent wires with the developers’ pay for the month and but the damn Ukrainian bank wouldn’t release the funds except with an eyedropper–a few drops at a time.  Apparently there was some kind of run on the bank, although it eventually eased.  Believe me, if there’s a chance your developers are not going to get their paycheck there is a very good chance they will be distracted.  But short of a Somalia-style breakdown of law and order, your software developers are going to need to continue to eat and pay their rents, just like we all do, and so they are going to show up for work.  So if this kind of thing happens, you might lose a little productivity, but you aren’t going to go out of business.  Beyond lost productivity, what senior management tends to fear most is a Russian-style impounding of their business.   But if you’re doing software development, there’s not that much they can impound.  Your computers?  So what?  They’re replaceable.  Your source code?  Big deal, it’s stored in a revision control system sitting in your home country.  Your IP?  What are they going to do with it?  Software is not like some manufactured good, so it’s not likely you’ll discover (as some manufacturers operating in China have) that your local business partner is now selling your own product, in competition with you.  So as opposed to an oil field, or a manufacturing plant, or some other business that has a lot of fixed assets, there isn’t that much the local Oligarch or government crony can grab from you if you’re doing offshore development.  What’s important to you is the team you’ve built and the knowledge in their heads, not some fixed assets.

Legal System — It’s nice if the legal and tax systems in a country are well-understood, and are not a moving target.  Unfortunately there does seem to be an inverse correlation between good legal and tax systems, and the cost of software developers.  I had a chance to visit Chile a few months ago, and by all accounts it is a bit more expensive than other countries in South America, but it might be worth it if you can’t tolerate red tape and corruption, since their legal system is considered to be quite non-corrupt and efficient.  Americans might be surprised to learn that, according to Transparency International, Chile is slightly less corrupt than the USA.  Similarly, Poland and the Czech Republic, as EU countries, will cost you more for developers than non-EU Eastern European countries like Ukraine or Belarus.  So you need to decide if that’s worth it–where on this continuum of legal and tax system effectiveness and stability you want to be, because you’re probably going to need to pay more to be in a more transparent and stable country.   If that will let you sleep better at nights, it might be worth the extra money to you.  Now, when I talk about the legal system in a country, I’m not just talking about suing somebody, or whether the system is corrupt.  We should also consider the taxation system.  You don’t need to worry about this if you’re working with an outsourcing company—that’s their problem (although if their taxes go up suddenly it will become your problem soon enough when they try to pass those costs through to you).  But if you want to create a “captive” subsidiary in the country, it’s helpful if it’s very clear what taxes need to be paid and how to pay these.  Check out PwC’s ranking of various countries on the ease with which companies can pay their taxes.  It turns out to be extremely difficult in some of the better-known offshoring countries just to pay your bloody taxes!  India is 147th on ease of paying your corporate taxes, requiring 33 different tax payments per year!  That’s a lot of bureaucracy, and most of the countries in Eastern Europe do very badly too.  For much of my career I’ve worked with captive development sites, but this is a great argument for dealing with an offshore service provider—let them deal with the taxman 33 times per year!  You’ve got better things to do.


Offshore Software Development – Gartner’s Top 30 Countries

If you’ve read some of my prior posts (such as this one), you know that I believe that the market for software development talent is now a global market.  That doesn’t mean that all development is moving offshore.  But it means that we have a shortage of good people in the US and if your company is above a certain size and scale, you should consider using offshore resources due to cost, and also due to the availability of good resources offshore.

Assuming you’re convinced of a need to do some software development work offshore, where do you go?  CIO’s are getting emails from random offshore service providers every week, if not every day, and it’s pretty difficult for them to distinguish among these providers.  They’re all offering what seem to be similar services at similar prices.  Normally when I approach a problem like this, I like to adopt some kind of framework to help me think about the problem.

Gartner has a framework for ranking the top destinations for offshore IT services.  There are some things I don’t like about the Gartner framework.  Analysts such as Gartner tend to be more focused on Business Process Outsourcing (BPO), rather than offshore software development.  So my approach will be to start with the Gartner framework, but I’ll modify it to focus on the factors that are most important for creating an offshore development team of from, say, 20-100 people.

Let’s take a look at Gartner’s list of top 30 destinations for offshore services:

  • Americas: Argentina, Brazil, Chile, Colombia, Costa Rica, Mexico, Panama and Peru.
  • Asia/Pacific: Bangladesh, China, India, Indonesia, Malaysia, the Philippines, Sri Lanka, Thailand and Vietnam.
  • Europe, the Middle East and Africa (EMEA): Bulgaria, the Czech Republic, Egypt, Hungary, Mauritius, Morocco, Poland, Romania, Russia, Slovakia, South Africa, Turkey and Ukraine.

If you are a Gartner subscriber, you can request individual reports on the various countries on Gartner’s list.

Let’s take a look at the countries on the list.  It’s evident that some of these countries aren’t on the list due to their software development prowess.  Bangladesh?  Costa Rica?  Panama?  Peru?  Seriously?  None of these nations are creating great software developers, en masse.  So maybe you could create a small team in one of those countries, but I don’t think it would scale beyond a few people.  What about the Philippines?  It’s a great option for BPO, due to the American-style English you can find there, and they have a large population, so one would certainly expect that at least some of them could be great software developers.  But most people I talk to just don’t think the Philippines education system is world class at producing large quantities of strong developers.  I had a chance to talk to a Philippines-based service provider recently, and while they felt they could provide good IT support resources (e.g., off-hours help desk), they shied away from the opportunity to bid on creating a large development team there.

Some of Gartner’s calls have been amazingly bad in the past.  Back in December 2010, they announced their updated top 30 IT offshoring destinations, which included Egypt.  EGYPT?  YOU GUYS CAN”T BE SERIOUS!  Not only is Egypt a political pressure cooker with an unclear future, but they shut down the bloody internet there for a few days back in 2011, just one month after making the Gartner list!  What if you had some production problems and the key engineer you needed couldn’t be reached because the Internet was shut down?  I think that’s enough to take Egypt off of your list, unless perhaps you are doing some work that’s specific to the Middle-East market.

Now, having had a little fun at Gartner’s expense, I should say that (by riding upon a client’s Gartner subscription) I had a chance to talk to a Gartner offshoring analyst about a client’s offshoring plans a few months ago.  Gartner wouldn’t advise you to just blindly go to the #1 destination (which by the numbers was India).  They view the list of top destinations as just a starting point to spur your thinking, just as we’re doing here on this blog.  In my next post I’ll look at the factors that Gartner uses to grade the various countries, and I’ll critique some of these factors.


Snarky Comments on Cloud SLAs (Part II)

In my last post I wrote about the problems with Cloud Service Level Agreements (SLAs), and why they tend to be less useful than one would like.  OK, so what can you do about it?

1) Ask Questions — If your service provider offers an SLA, study it and ask questions.  “Is that all you can promise?  What’s your historical performance?  Can you show me some documentation of that from your monitoring tools?  Have you had any downtime incidents in the last five years?  What were the root causes, and what did you do to address those?”  The discussion will be useful in filling in your picture of their capabilities.  In the end, an SLA is just a number (or a few numbers), and it doesn’t give you much insight into the provider’s ability to deliver.  An SLA is a bit like a car warranty—it’s nice to know you have it, but your real hope is that it won’t be needed and the thing won’t have any significant breakdowns.

2) Try to Negotiate — Good luck on this, but give it a try.  I’ve found that, normally, the salesman doesn’t have the discretion to change the penalties in the SLA.  However, if the vendor doesn’t offer you an SLA, just asking might get you somewhere.  In the logistics software space, there is a company called SMC3 that creates a system used for calculating the rate of a Less than Truck Load (LTL) shipment, and most companies in the industry use this product.  They recently converted from a traditional licensed software model to a cloud model.  I was at their conference in Atlanta a few days ago, and asked about their SLA policy.  They don’t promise an out-of-the-box SLA, but they say that they’ll negotiate one on occasion.  So it seems that if you are a big enough customer to them, you’ve got a chance of getting an SLA in writing.  You’re not doing this to negotiate the SLA up to a level with which the vendor isn’t comfortable, because that isn’t going to change their real-world ability to deliver on the SLA.  But you want to understand what they can deliver, and get comfortable (or not) with this level of performance, assuming the vendor’s service is critical to your business.  In the case of SMC3, if you are moving LTL shipments for a living and you need to quote prices to your customers, then you’re really depending on SMC3 to deliver, because if their service goes down you can’t provide price quotes to customers.  So you owe it to yourself to understand their capabilities.

3) Look at Audit Reports – If the vendor has been audited (e.g., using the SAS 70 standard) then you should get a copy of the audit report.  Read it.  DO NOT just treat this as a check mark—“great, they passed an audit.”  Look where that got investors in Enron and Worldcom.  Like most standards of this nature, the audit “control objectives” for SAS 70 are essentially whatever the cloud vendor says they are.  So the cloud vendor tells the audit firm their control objectives, and the auditor audits to check that there is a reasonable assurance they would achieve those objectives (the auditor will state very clearly, though, that this is not a guarantee).  The auditor will make sure the business objectives that the cloud vendor uses are not completely stupid, but I find that a lot of people assume (wrongly) that SAS 70 is some kind of seal of approval that the vendor is doing “all the right things”.  SAS 70, for example, says nothing about providing disaster recovery (DR) capabilities.  A vendor can provide zero DR, and pass an SAS 70 audit with flying colors, simply by not stating any control objectives regarding DR, but showing compliance with whatever control objectives they do have.

4) Do Your Own Audit — While it might make you feel good to host your systems in-house, because you feel you have more control, chances are that you can’t do as good a job as a strong cloud vendor.  You probably know someone who feels comfortable driving a car, since they control it, but who doesn’t like flying in an airplane because they have to trust the pilot, even though flying is statistically safer than driving a car.  So maybe you should find out if your cloud vendor knows how to “run an airline”.  If the service provider is critical to you, arrange an audit where you visit them and understand their capabilities.  Are they doing the basics?  Do they have multiple power supplies on each server, or only on some of them?  Do they have multiple NICs on every server?  Ask them to show you.  Do they have current maintenance contracts with their key vendors (server hardware, network hardware, etc.) or did they “forget” to renew them?  How fast would that vendor promise to rush in a part required to fix their box if it went down?  Next business day? Hmmmm.  One of my customers once did an audit like this on my data center, and while it was painful at the time, ultimately it was beneficial for me and my team, and for the customer, since they had a much deeper knowledge of our real capabilities, and in the end they grew comfortable that we knew what we were doing.

Tagged ,

Snarky Comments on Cloud SLAs

When I read articles about contracting with cloud service providers (or other IT service providers such as colocation providers) often I see a lot of emphasis on Service Level Agreements (SLAs).  That’s the wisdom that they offer you: make sure you have a solid SLA with your cloud service provider.  The only problem with SLAs is that while, theoretically, they should be a useful tool for managing your cloud vendors, in practice they are a bit weak.   Here’s why:

1) The standard penalties you are likely to get in your contract with a cloud vendor if they miss their SLA are miniscule compared with the pain and suffering you are going to experience if the service goes down.  One cloud provider with whom I’m familiar (I won’t name them, but their SLA is pretty typical) will provide a 5% credit if their uptime is between 99-99.5% (their SLA guarantee is 99.5%).  So let’s say you are spending $5000 per month with that service provider, and that during one month they are down for 7 hours, and you can’t service your customers for most of a business day.  What do they give you?  Let’s see….a month with 31 days has 744 hours in it.  So if they go down for 7 hours, we’re just above 99% uptime.  So they owe you 5% of $5,000, which is $250.  ARE YOU KIDDING ME???  Your business is out one day’s revenues, the CEO is on the phone asking to speak with “shithead”, and your cloud service provider is going to give you $250?

2) Typically, cloud vendors lowball the SLA.  A typical number you’ll see in the marketplace now is uptime SLAs in the range of 99.5%.  So over the course of a year, they could be down for…365*0.005 = 1.825 days?  There’s a good chance I could hit 99.5% hosting your software on my laptop.   (Well, as long as I promise not to fire up iTunes, which seems to crash my Windows 7 laptop.  Yeah, yeah, I know.  It rocks on a Mac.)  By the way, that 99.5% target doesn’t include scheduled downtime.  So as long as they announce the downtime ahead of time, it doesn’t count.  By the way, ask your vendors how much notice they provide.  Some vendors only give about a day’s notice for maintenance windows, which makes it difficult if you have users that use the system during off-hours, when the vendor is doing the maintenance.  So anyway, the cloud vendors just aren’t setting very aggressive SLAs.  If you ask the vendor, they’ll probably admit that they are lowballing the SLA, or at least they’ll say, “Well, our long run average is certainly better than that.”

3) Some cloud vendors won’t even provide an SLA.  Salesforce.com is an example.  At least they won’t give you one if you’re not a mega-customer.  And apparently even for big customers they try to avoid doing so, to the point where according to some sources (http://www.online-crm.com/sla.htm), they won’t even call you back if you call and ask for an SLA!  So they are really just saying “look, we think our service level is good enough, but we’re not going to guarantee it, so just try it out and if you aren’t happy, then just leave.”  For certain kinds of services that’s probably good enough, but for “mission critical” services, it’s not a very reasonable answer.  One thing Salesforce does that I like a lot, though, is give you transparency to their current system status, and recent history.  Check this out:  http://trust.salesforce.com/trust/status/.  So you can have some feeling that they’ll be honest with you if there is an issue.  However, they don’t provide data on their long-run historical performance and they aren’t making you any promises.

So what do you need to do to?  Forget about SLAs?  Not entirely.  I’ll talk about some ideas for managing SLAs in my next post (people have been telling me I put too much information in my posts and I need to chop it up!)

Tagged ,

Is there a US Developer Shortage? Why Going Offshore is Just a Matter of Time

If you have a small development team of a dozen people, or fewer, you can probably do fine developing all of your software onshore.  It might even be the right call, provided you can afford it, since at that point you might just be figuring out what your product really is, and it’s better to work with developers who sit with “the business” and are able to react quickly to what the business is seeing in the marketplace.  Or you might need some specialized skills that just aren’t available offshore.  Sometimes if you’re working with the latest of the latest technologies, or something a bit arcane, then you might have trouble finding that skill set offshore.  Or, if you need some awareness of the business or social environment in your home country, then people at home would have a better intuitive sense of what you’re trying to accomplish.  For example, if you’re building a social media product for bar-hoppers in US cities, you’re probably best to develop version 1.0 as a “Lean Startup” with 2-3 developers in a major US city, since you’re going to have a tough time explaining to offshore developers exactly what your product vision is.  And if you try to explain it, they’ll most likely give you a product that doesn’t fit the bill, and tell you that “you kept changing the requirements” (and they’d probably be correct about that).

But at a certain point in your company’s life, assuming you are above a certain scale and size, you’ve most likely got to do some work offshore.  Of course there’s a cost argument.  Loaded costs for developers at an outsourcing shop in India are around 50% of the costs for comparable people in the US.  And if you have the time and knowledge to form a “captive” offshore development shop, you might save more than that.  But apart from cost, it seems there just aren’t enough software developers in the US to meet the demand anymore.  There’s a contrarian point of view regarding this, and you can hear it from commentators like Ron Hira, who claims that actually there is no shortage of developers in the US, and when companies go offshore they are just looking for low costs and there’s no supply issue when it comes to developer talent.  In some of his other articles, Hira makes some good points about the H1B visa program in the US which, while generally positive for the US, has been abused by some of the IT outsourcing firms to bring in undistinguished talents, and pay them below the US norm.  But in this case, I think Hira’s work suffers because he lumps all IT workers into one big bucket.  I’m not talking about a shortage of generic IT workers.  But if you need to hire highly skilled software developers, who are current in the latest web and mobile technologies, and who are the best in the world at what they do, there is data that shows that there is a shortage in the US.

The Wall Street Journal recently reported that there are over 150,000 openings for software developers in the US right now.  Intuitively that sounds a bit high to me, but there’s no doubt that there are major shortages in some key areas, such as mobile development—who isn’t thinking about their mobile strategy these days?  And in certain areas of the US, there just aren’t enough people in the region engaged in software development, and if you need to hire a few experienced Java developers, it turns out to be quite difficult.  If you’re looking for these people in quantity in the US, and your company is not called Facebook or Google, you might be fighting a losing battle.  So you might be better off offshore, where you can raise your wages a bit above market and attract some strong talent.

Is that the future of work in the US?  Is all of the software going to be built offshore, while those of us living in the US will all be “idea people” (and burger flippers and retail clerks, I suppose)?  Do we need to encourage more American kids to study “hard” subjects like science, engineering and math as their college majors, in order to make us more competitive as a nation?  One would think that the market would signal the need for more developers through rising salaries, and more people would respond to those signals and would study disciplines like computer engineering and computer science in college.  Good luck with that.  The problem is that the dogs (students?) just aren’t eating that dog food.  A recent study reported in the NY Times found that students drop out of these “hard” subjects in droves.  It’s not exactly clear why this is the case, but it seems to have something to do with these courses of study being something like the Bataan death march.  It seems for most American kids, the prospect of $60K+ starting salaries for entry level developer jobs just doesn’t justify having a shitty undergraduate experience, just when they are spreading their wings and getting out of the house.   Apparently President Obama set a goal of graduating an additional 10,000 engineers per year.  But if you believe the experts (at least the one guy quoted in the Times article), that’s just not gonna happen.  Even if we got to 10,000 additional engineers per year, and we peel away the civil and chemical engineers, and others who can’t help us close our 150,000 person software developer gap, then it would appear that our gap wouldn’t close for years anyway.

I don’t really have a realistic “three-step plan” to solve this software developer gap in the US, and I’m not aware that anyone else does either.  But I think it’s a real gap, so for the foreseeable future the only way US companies are going to get the developers they need is through a mixture of offshore and onshore software development.  So we might as well just admit this and figure out how to be great at managing mixed teams of onshore and offshore developers, probably using Agile-type processes.  I’m not of the belief that there’s no future for US developers.  It’s just the opposite—there’s plenty of demand for them.  But there just aren’t enough developers with the skills we need onshore, so companies above a certain scale and size need to think offshore.