The Rush, in Retrospect

Preparing for the meeting did not go well. Live, on-site, with client and end customer, and it could have been better.

Crud.

The actual demo was a success, but getting there was not. What happened was not fun. It was not fun for the client, whose expectations were not met. Not fun for our bosses, who dealt with the brunt of the fallout and had reputations on the line. Not fun for the dev team, who worked hard to get the demo even to the state it was in, plus felt the guilt for a less-than-stellar technology rollout.

Now that the smoke has cleared, the yelling and finger-waving has stopped, and blame has been passed around, we can take a look at what happened. Two questions need to be asked:

How did we get here?
How do we keep from happening again?

Getting here was the easy part. Because everything is fine, before it is not! Looking with hindsight, there were a few things that changed silently. Dates got pushed up a few weeks. Demo changed from a tech team demo to a customer meeting. Meeting turned from a demo into a pre-launch. Feature or two changed from the plan. That little technical hurdle that takes longer than planned.

Thinking everything was on track was easy, but as changes to expectations started trickling in from different places, they were not all in sync when the meeting rolled around. And that is how we got here.

The thing is, by then it was too late. There was not time for fallback plan. Strength of will or more management oversight cannot make anything happen faster, or even just fast enough. The yelling was endured, the plan was laid out, the weekend was worked and with a final push, the launch was successful.

Hooray?

The result of a successful launch is good, but getting there smoothly is a better goal. Since this was not smooth, how can we keep it from happening again? Ultimately it falls on us. The client will always be the client, so they may miss art delivery deadlines by weeks, set meetings without our input, and always ask more (for less). Might be annoying at best and infuriating at worst, but clients will be clients!

The point of process is for situations like these, when expectations are key. There were no defined requirements or checklists of what was needed at what date, just that "it should work and be awesome." Somewhere our software process broke down, which led to bad expectations. Running with a broken process is overly misleading too, so by not assessing things early we did not address issues soon enough. At each change in dates, we needed to have been very clear about what features were going to be ready, not a mix of "everything in the contract, just less, with different dates."

The other thing that would help things run smoother is a defined contract style. In broad terms, the types of contracts we have accepted are fixed price and retainer. Quick recap of each:

Fixed Price

• Defined set of features to deliver.
• Defined delivery dates.
• Set price

Retainer

• No specific features, but team is available to work
• Dates are defined for retaining purposes ("start of each month")
• Price is "per time" (hours worked, or monthly)

Fixed price is good when someone knows what they want, and requires a lot more up front planning. People always want fixed price (and want it to be lower!), but often do not take the time to define features. Good for fixed deadlines (trade show demo, bottom-line price for release), and often has some contention toward the end like "that feature was expected" "it obviously should work this way".

Retainer is a good model when people know they need work done, but cannot define what that is, or have a tendency to change their minds, or need to see it to know if it was "right." Usually things are smooth in the start and middle of a contract, and the problems come up at the end of a retainer, where people always question the effort without knowing how long it should have taken in the first place.

Back to the post at hand, we mixed the two. We were signed up for a fixed price with fixed deadlines, then changed the deadlines and were working like a retainer model. Most any change that was requested was started to be incorporated, which was OK until some people were thinking the original deadlines ("no problem, plenty of time!") and others were thinking the (new) demo dates. Now doing retainer-style work was not meeting specific and defined constraints, so while everyone was "busy", it was not focused on the important things.

The contract is now basically concluded, with time to spare! Learning from ourselves this time, we'll likely improve for the next. 

Prototype Rocks Are Better

A common analogy with my bosses is "Bring me a rock." This fictional conversation between an "idea person" and "engineer" would go something like:

Idea Person: "Bring me a rock."
Engineer: "Here is a rock."
IP: "It is not dark enough."
E: "Here is another rock."
IP: "Why is it so jagged?"
E: "You want a darker, smooth-ish rock?"
IP: "Yes, obviously."
E: "Here is a rock."
IP: "...it's too small."

We are not geologists, and this conversation is not really about rocks.

Ideas are not bad. Quite the opposite, they are necessary! People who come up with ideas are vital from an entire organization down to a small development team. New products or features were not written down long ago and rehashed; they were new ideas at some point and through many steps, and sometimes many iterations, became reality!

Ideas are hard to pin down, and sometimes they are fluid. When a person has a great idea, there are steps involved to make that idea real. Many steps. Rambling off a few sentences of the idea and thinking it will be done to any vague expectation of success is hard to achieve. Especially to an engineer who needs (lots of) details.

And yet, this happens all the time. Which is normal in a lot of cases, and idea people and engineers work together to make it happen. It's called "design."

This  situation becomes a problem when the idea person and the engineer are not working together. Suddenly the idea person wonders why their simple idea is not done yet, and the engineer is frustrated at wasting time on undefined "requirements." Two or three iterations of "Bring me a rock" can wear someone out! Compound this with an idea person that changes their idea before the engineer shows them anything, already invalidating some of that engineer's work! Aaaaarrrgh!

That why you need a prototype.

Having a prototype type is the place to really start a conversion with those idea-person-types. It is tangible. It is something to talk to and point at. The dev team needs to get that prototype ready as fast as they are able (keeping in mind their process and maintainability goals), and the "idea people" need to wait until the prototype is ready before making any new suggestions or changes. Unless something is drastically wrong, or some data proves otherwise, stick with the prototype and discuss when it is ready.

Beyond shortening the "bring me a rock" exchange, a prototype clarifies ideas. When five people leave the idea meeting thinking of a rock, more often than not they are picturing five different rocks! Looking together at a prototype may result in a long list of changes, but the changes are defined and everyone is clear on what is being done and what to expect as a result.

Prototypes are also handy when people do not even know the type of rock they want You might explain the rock five different ways, but until someone sees it, it might as well not exist!

E: "It's a round, smooth, gray rock about the size of a grapefruit."
IP: "I've never seen that before."
E: "I haven't built it yet. Just try to imagine it."
IP: <blank stare/>

Great explanations and ideas will only go so far. At some point it needs to be tangible. And once it is tangible, people can actually engage in the conversation.

Hopefully bringing rocks and asking for rocks will be part of you future. Just remember that prototype type rocks are better. For everyone.

git Permission Denied after reboot

Common issue that I just dealt with again, but wanted to write it down for that time I won't have it memorized.

My machine's git account is tied to the rsa key for github. Company switched to Bitbucket, I created another key to keep the accounts separate, and now whenever I restart my machine git gives me an error like:

Permission denied (publickey).fatal: Could not read from remote repository.
Please make sure you have the correct access rightsand the repository exists.

Turns out I need to add the new key to ssh because it loses track. Some folks have suggested putting this in their bashrc so they don't need to type, but I don't restart the machine too often and typed this post instead.

ssh-add ~/.ssh/id_rsabb

The rsa key is in the .ssh directory, but needed to be added to the console. "id_rsabb" was the name I chose for my bitbucket rsa key. Simple command, enter my key's password, and git + ssh are working together again. 

AWS SNS.publish Extended Message Data

Using AWS SNS in NodeJS to send Push Notifications, and we wanted to append extra data to the payloads. AWS has good docs for the plain text message, but trying to use a JSON Message object in the sns.publish call saw an error like:

016-07-12T16:19:09.713Z - error: InvalidParameterType: Expected params.Message to be a string

After some searching, I found the answer in an AWS forum dated back to 2013. Thought I would repost here for easier reference to myself.

The trick was those extra JSON.stringify calls, once to turn the platform messages into strings, and once to turn the entire Message object into a string. Seems like some odd parsing, but I have received notifications on an Android phone, so at least GCM likes it.

Redis Ordered Set WITHSCORES into JSON Object (NodeJS)

A note for myself before I stop using this ordered set.

Redis Ordered Sets can return the members of a set with their scores. Its an option on the zrange command, so to get all the values in an ordered set with their scores:

`redis.client.zrange( key, 0, -1, 'WITHSCORES')`. 

Of course, the returned array is pretty specific, something like:

user1 135000 user2 384000 . . .

Using the handy lodash library, I wanted to keep track of this little hack:

//this is a synchronous operation and would block the event loopvar testScores = _.fromPairs(_.chunk(testValuePairs, 2));

I've seen it done with for-loops that skip every other array item, but I liked the single line of this approach. It does go through the array twice, once to turn into paired array, and once through that array to create the object, but for my purposes the array sizes were going to be small and the succinctness was useful.

Team Focus

One of the intangibles of software teams, or any team that is working together on something, is team focus. Focus is vital to keep everything flowing and to meeting expectations, but a lot harder to turn into a metric or graph because it is a soft measure. It is not quantifiable. It is almost a . . .  feeling!

Team Focus is how well a team works together, towards the same goal, and are self-motivated to achieve that goal.

Focus among individuals varies and some need a little more coaching than others. Some will get into "the zone" quickly and may even seem anti-social. Some are too addicted to their phones or Facebook, and that's just an employment concern. Some need the instructions once and it gets done and some need reminding more often, and it still gets done! However, these are mostly all personality traits. While a team should have a few different personalities to avoid freaking out the other departments a la The Stepford Wives or The Borg, an individual's ability to focus is just part of the equation.

The team's focus cannot be measured in the way we can measure tickets, sprints or SLOC. You won't see a graph or color chart indicator. You might think you would be able to deduce your team's focus based on the measurements of your process, but measuring the wrong things or good planning could be factors there. Just look at the components of that "team focus" definition above and see how you would describe your team.

"Works Together"

How your team works together is paramount to being focused, and it is affected by the company culture and personalities. Teammates who are willing to help and explain make a team work better together. Questions are not a bad thing. Whiteboard sessions are a group effort and all opinions are heard. People smile. There are inside jokes at the office about non-project, and non-people things.

To many, it is obvious when a team works well together, and is not "just working." They actually get along, and that is part of keeping them focused.

• Does your team have a loud-mouth or know-it-all? That does not build morale, and it is already a factor as to why people aren't staying on your team.
• Are coding opinions bashed as inferior? If the "right way" is not written down (and, if possible, automated by tooling), just deal with it. You can still read the code.
• Is there any blame or names at bug review, or deploy time, or at demos? Double-check the review process and steps prior to release, because the team released the software.
• Does one part of the team have the inside track for project decisions, and the others are only informed later? Be careful if these are not documented roles, because saying "this is how it is now" leaves one part of your team on the outside. 
• Are there two sets of standards? Billy can skip writing a few tests because his work is important and needed right now, but Bobby gets dinged for skipping tests. Susie is made to follow the coding conventions during review, while Peggy doesn't run the auto-formatter. Watch out for this, it is divisive and will chip away at the team focus.

"Toward the Same Goal"

The team knows what it is working on, and you have a process that proves it.  There is business buy-in, so the team knows it's the right thing to work on. The stakeholders will be happy, so the team expects some kudos. Everyone knows where their piece fits into the current sprint/milestone/demo, and why that is important for the future of the project.  This, all wrapped together, is the object of the team's focus.

Know the priorities of your goal. Is it better to build performant or functioning software this cycle? Should the code be good or perfect? Tests too? Can your schedule accommodate "all of the above"? Different approaches to those questions can still look like the "same goal" but if each person has a different answer to the questions, there is a lack of focus.

Also, and this may sound obvious, but do not change your goals mid-sprint/cycle/sentence! If a sprint is defined, don't add to it after starting. If a feature is planned and started, don't add to it until the next planning phase. Changing goals are ludicrous to track. Your team cannot anticipate when new work will suddenly be added. No one can answer the question of "When will it be ready?" The dev team just sees arbitrary buckets of tasks, that can change or be added to, which certainly does not help them focus, and sometimes crushes morale.

"Self-Motivated to Achieve"

Did you team catch the fire? Do they believe in the project? Are they excited to show off the demo? Can they hang their hat on a day/week/sprint worth of work? Would they call it their own?

Like many things, people make more an effort and achieve better results when they are passionate about what they are doing. Being a creative bunch, software developers need a little room to grow their ideas, so don't force opinions all the time. Make important tasks, not just filler or "grunt work"; even test writing can be meaningful. Open up parts of the process and operations to others, to grow their experience and to give more responsibility. Be careful of someone hogging all the hard-stuff/glory/blame - this is a team. Compliment a nicely developed feature, or a sweet algorithm. Tangibly celebrate team accomplishments! (FYI - an email barely counts.)

The more involved someone feels in the team, and in the process, the more meaningful their work becomes, and the more ownership they will take. And they will be self-motivated to reach team goals and achieve higher standards.


While one cannot use their software process to measure team focus, the focus of a team will directly impact a team's results. Hard to measure yet highly important. Strive to keep your team working together toward the same goal, and self-motivated to achieve that goal.

The Point of Process - Part 2

Once the point of process is understood, it is time to measure your current process. This step is important because the point of doing any software process is not "is it good Agile"" or "did we follow methodology X correctly?". The point is: does it work.

A contract of the past is what spurred on this article. We were working tightly with another company, completely ingrained in their process, and my boss would ask me, "How's it going?" My response was mixed, because on the one hand I knew what I was assigned to work and was working on that. That makes a boss happy. But my boss, being a stakeholder in the work being done, was really asking those two questions from Part 1 so he could evaluate the current contract and plan for new ones. I'd tell him, "I'm doing work, but I have no idea where it's going and when we'll get there!" It was a software process that was missing the point.

The same questions that the stakeholders are asking are the same questions to use to measure one's software process. If you have an answer, the process is working, and if it takes more than 2 minutes generate or explain the answers, there is a problem.

Where is the software going?

Every member of the team should know the answer to this question. If one is a Team Lead or Product Manager, the answer needs to go both ways. It goes up to the stakeholders, like Business Dev, Marketing and the C-Suite. If they are not directly planning what the software must be capable of, they at least need to be made aware what is coming next in terms of features and improvements. 

This question must flow down to the devs as well. If they do not know where the software is going,  motivation drops and future features are not planned for. Keeping an air-gap between business and dev stifles ideas from those that know the technology the best, and lowers the meaning of work they are doing to simply "doing what they're told."

When looking at your software process, each member on the team must be able to know, quickly and easily, where the software is going. 

• Can anyone look at the list/board/spreadsheet/diagram and know what is coming next? 
• Can they see where their work fits in right now? 
• Is there a place to write down new ideas? 
• Are the milestones front and center? 
• Can the reason of the feature (or entire project) be summed up in two sentences that give meaning to the devs and makes sense to the business side? 
• What will come next, after this cycle of development?

Use your process to understand where your software is going.

When will it be ready? 

The schedule is vital to a business, so the development team cannot simply stall for time, or make excuses like, "it's ready when it's ready." Demos, marketing, trade shows, and everyone's paycheck is resting on these dates. Any slips need to be communicated. (Any early finishes should be celebrated!) 

A memorable quote by Walt Disney, which I like to repeat, is "Everyone needs deadlines." Walt goes on, but that's the important part. The software schedule sets everyones expectations of "done". And without it there is confusion, miscommunications and unmet expectations. Developers need to fit their work within the timeframe, and cannot go off the path just "because" ("because this new way looks cooler", "because I want to refactor now"). Estimation is an important skill for any team, and fitting what must be done within time bounds is part of the planning process, not the middle or end of the execution phase.

The schedule has to be clear, and your process must make it clear. Having business side stay out of your hair until the software is supposed to be done, and then breaking down the door after a missed date is terrible. Don't work like that! Use the process to quickly and easily inform the business side and guide the development side. 

• When is this cycle of software development going to be done?
• How close is the team to being done? How much more time is needed? 
• When is the code frozen (if not continuous)? When will QA look at it? When will customer see it?
• Is the team currently on track? How accurate were previous estimations? 
• When is the next major milestone?

Use your process to understand when the software will be ready. 

If You're Happy and You Know It

What is the ultimate measure of your software process? Whether or not the stakeholders are happy! That's it. If they are unhappy, look in to ways to improve the process to better answer the two questions. (Hint: this likely involves bringing them into your process, not pushing them away). If they were happy, then great! And take a little time to see how to make your life easier.

Stakeholders looking at your software might not be very technical. Metrics and procedures mean very little to them. When they evaluate they are looking at two things:

1. Did you do what you said you were going to do?
2. Was it ready when you said it was going to be ready? 

Be able to answer "yes" to those two questions and you will have some happy stakeholders.

But you knew that was coming, right? Because your software process, quickly and easily, has been answering those two questions all along, so everyone knew exactly where the software was going and when it was going to be ready.

That's the whole point of all this process, and that's how you know it worked.