Software Development: Moving from Art to Science

Have you read Moneyball? The book is about how Billy Beane, the general manager of the Oakland Athletics, revolutionized the way we evaluate baseball players in order to create better teams with less money (maximize quality while minimize cost). The premise is that the collective wisdom in baseball is based on antiquated statistics that were defined a hundred years ago based on what was easy to measure rather than what could be measured; baseball management was living in the 19th century. Billy Beane believed that by using 21st century statistical analysis, he could create a far better team with far less money. When he put his ideas to the test, his team became one of the best in baseball with one of the lowest team salaries in the game.

I believe in software development, we are still living with antiquated and immature metrics and that there has to be a better way. Let me give you an example. Take project estimation and staffing. In a professional services firm you often have to bid on projects without any idea of who will be on the development team if you win the bid. So what do you do? You decompose the project into discrete tasks, assign time estimates for task completion, do a resource leveling and determine project duration and cost all while using some notion of an 'average' developer. Weeks or months go by before you are notified that you have won the bid and then you begin staffing the project. Here is where the challenge begins.

When you did your task estimates, you assumed you would use an average developer but what metric(s) did you use for the classification? Assume for a minute that you have a team that specializes in the technology you bid, is an average developer a Junior Developer or a Senior Developer? We all know that there is a great range in developer skills yet how do we rank these skills? Is a developer a 10 or a 1 and what would a 10 or a 1 mean? If you bid the project for an average person, is average a 5, a 7 or an 8? Does it take into account quality, clarity and supportability of code or the maturity of the developer?

Actually, how we typically judge a developer is highly subjective, it is with titles like Junior Developer, Developer or Senior Developer. Within each job class, we may have salary bands and the bands may reflect relative skill level or they may reflect different managerial judgment. What is needed is a more accurate measure of a developers skill set. What is needed is a productivity rank for each developer.

Let me walk you through an albeit contrived example to show you the complexity of the problem and the need for a better solution. Let's assume that you have devised a way to grade a developer and have determined that there are 10 different productivity levels. I've often heard that a great developer could be as much as 10 times more efficient than a bad developer, so let's assume that the levels are relatively incremental and that an average developer is defined as a 5. This looks something like:


Now, let's overlay on top of this a typical classification scheme ranging from a junior developer to a senior architect. I'll add a second Y axis for costs an will add in a notional curve for salary and billing rates.


So for a constant level of effort for a project, you can then staff the project in many ways. For example:


Here, the base resource, ranging in capabilities from a 4 to an 8, is augmented with additional staff to complete the work. Mix 1 adds four additional junior resources (rated 3) while Mix 2 leverages five trainees (level 1's), and two junior resources (level 2 and level 4). Mix 3 suggest that the effort could be completed by 2 slightly better than average developers (level 6), while Mix 4 and 5 look at using more senior resources teamed with a junior resource. As the different mixes imply, there are many different ways to staff the same level of effort. A challenge for project managers is that the task breakdown likely lists this effort as 2.5 FTE (Full Time Equivalent). Without knowing the skill set of the staff, 2.5 is a meaningless number. If you put 3 resources on to the project who were all 3's you would not have enough ability to complete the project. Of course, we typically blame the manager in this situation for clearly (s)he had a enough resources (3 being great than 2.5) to deliver on schedule.

Without knowing the productivity factor for your developers, it is dubious to say a task is 2.5FTE and even harder to staff it simply as 2.5 FTE. It is even harder to assure you have the right mix to optimize your gross margins vs. code quality. In the above example, I've put some relative numbers to the task based on the graph:


For this cost structure, I've created a nominal Fixed Bid as the median of all the bid prices. Gross margin is calculated as a percentage of the bid price. I've also computed relative average bill rates and total man hours (as a percent of the 'standard' which I've used as Mix 3).


For the same project and the same level of effort, you can have a 4x increase in man hours and average bill rates. Gross margins are optimal with senior resources though as Mix 4 indicates, there are tradeoffs that help with average bill rates and man hours vs. gross margin.

Though it pains me to say it, software development is still more art than science. Without accurate productivity ratings for each developer, the industry will continually struggle to fill projects defined by FTE.

How does your company resolve this? What metrics do you use to ensure accurate staffing?

An American Working in an Indian Company - Part 2

In an earlier article, An American Working in an Indian Company - Part 1, I talked about the first half of a presentation the CEO gave to me to prepare me for working with the staff in India. The second half of the presentation focused on communication differences. Taking the time to understand these differences can really improve your ability to work offshore.

The discussion on communication differences was largely based on the work done by Edward T. Hall in Silent Language.The basic concept is that much of our communication is non-verbal which consistently follows cultural and linguistic patterns, just as spoken and written communication does. The major difference in non-verbal communication is that it is mostly subconscious. A real challenge in working with people from a different culture is that non-verbal communication is hard to pick-up. This is even harder when all the communication is virtual. Even with a high quality VTC, it is hard to see the nuances of non-verbal communication and it is even harder if you do not know what to look for.

The presentation covered three principle categories of communication differences:

• Spatial Factors
• Time Formats
• Context Levels

Spatial Factors
Spatial factors are territorial or personal. In the US we all have our own territory, like our office. Our personal office is a static space, it is our space, it is the place we go to get away from everyone else in the office. Our office creates a sense of 'home' and we can feel violated if we find someone in our office unexpectedly. In India, there is very little sense of territorial space. Offices are more open and people share spaces more frequently. In the US, you can read people by examining how they set up their office (their territorial space). Do they have their back to the doorway or do they face the entrance? Do they have space for visitors to stand or sit (shared space) and where is it in proximity to where they sit? In India, there is a lower sense of importance on territorial space and hence it is harder to read peoples preferences. There is a good summary of this online at Haverford College.

Personal space, and the size of ones' personal space, also differs greatly and this can really create communication challenges. We all have our own opinions on the appropriate distance we should be from one another when we talk. We have an outer space for communication and an inner space for intimacy. If someone gets too close we feel they are threatening, dominating, or worse, that they have inappropriate intentions. Similarly, if someone is too far away we feel they are standoffish and have a lack of interest. We in the US have a very rigid definition of our personal space and, relatively speaking, it tends to be very large. In India, this space is much smaller and is much less rigid.

My first experience with 'violations' to my personal space happened on my first trip to India. I was on a plane, flying from Amsterdam to Hyderabad. I was one of three or four non-Indian's on the plane. When I started to put my luggage in the overhead, I noticed that I got bumped 3 or 4 times by people walking by. I was pretty taken aback as not one of them apologized. I though about this as the flight progressed and when the flight landed I decided to watch the other passengers to see how they interacted. Sure enough, everyone stood up at the same time and tried to grab their luggage. Everyone jockeyed for position and tried to stand in the aisle. I was surprised at how close everyone stood to each other. When a plane lands in the US most of the people with an aisle seat will stand up, but the rest of the people will wait for the aisle to clear before they stand. We require more personal space and therefore give each other a wider berth.

When I went to the office the next day and met with the staff, I was told I should use an executive's office because he was out for the week but after I sat there for a few minutes, I felt uncomfortable setting up shop in someone's space so I moved to a conference room and made that home for the week. I got a lot of odd looks and realized later that my sense of territorial space was at odds with the staffs and that I had made a mistake by claiming the conference room. To them it looked like I wanted more space while to me, it was simply that I didn't want to violate someone else's space.

Time Formats
Time formats describe how we work based on how we view time. Are we always on a tight schedule and watch the clock, or do we feel time is renewable allowing us to make time for what is currently important? The former is monochronic while the latter is polychronic. In the US, we tend to be monochronic and India tends to be polychronic. This creates a significant cultural challenge. In the US, we like to create a plan, divide it into discrete tasks, assign the tasks, focus on each task and complete them one at a time so that we achieve our deadlines, which are set in stone. In India, plans change all the time so they work on many tasks at once, often ignoring prioritization, they expect interruptions and distractions, and treat deadlines more line guidelines.

I can remember working on our first major account. We had 40 people on the project and the customer had requested slightly aggressive deadlines. I had impressed on the project manager the importance of the account and I walked him through the culture presentation so that he would understand the customer better. The project manager took this to heart and really focused the team in India, stressing the importance of meeting our dates. At first, the team was supportive but as the project progressed, the pace and the commitment needed to complete the project on schedule really started to frustrate them. They kept asking for extensions and couldn't understand why more time could not be granted. The project manager wanted to ensure we hit the dates and started working the team harder and the resentment grew. Though we hit all of our deadlines, the team, to a person, asked that we fire the project manager. From my perspective, he did a great job, but I had to really fight for him with the senior management team in India. In the end, we lost him for the team refused to work with him again. The result of his efforts helped us land a larger piece of follow-on work with the same customer. So what was appreciated in the US cost him his job in India.

Context Levels
When we talk, the context level implies how much background or additional information we give in our discussions to make sure we are aligned. Communication can be classified as highly contextual, in which case much is assumed to be understood and little needs to be said, to less contextual in which case less is understood so more must be said. The US is a low context culture while India is a high context culture. This means that in general, Americans tend to over communicate while Indians, in general, under communicate.

I learned about context the hard way. We had a customer in the construction insurance space and I brought a team from India to work with the customer on requirements. The team had experience working on insurance projects so I didn't think to prep the customer in any special way. In the requirements meeting, the customer focused on all the hard and complex issues that they felt had to be done right. The team did a good job of capturing the requirements for the complex scenarios and then went back to India and began the development. The project was only a three month engagement About two months in to it we showed the customer a beta version. That was when things began to unravel. The team actually had gotten almost every complex case right, but they had omitted, or gotten wrong, all of the simple cases. Needless to day, the customer got very frustrated. They could not understand how we had missed all the simple scenarios while at the same time we had the complex ones correct. I was also at a loss. The reason the team missed all of the basics was because they didn't have the context to work from; insurance in India is simply too different from the US. They had assumed that there were only complex cases. One engineer commented that I had told him that Americans over communicate everything and that the customer never mentioned the simple cases, so they never thought to ask.

Communication is hard enough when you sit next to each other and have the same cultural background. When your customer (or provider) sits 12,000 miles away, communication becomes much more challenging. Taking the time to understand the cultural differences in communication styles can really improve your ability to communicate and to successfully work together.