The Point of Process - Part 2

Once the point of process is understood, it is time to measure your current process. This step is important because the point of doing any software process is not "is it good Agile"" or "did we follow methodology X correctly?". The point is: does it work.

A contract of the past is what spurred on this article. We were working tightly with another company, completely ingrained in their process, and my boss would ask me, "How's it going?" My response was mixed, because on the one hand I knew what I was assigned to work and was working on that. That makes a boss happy. But my boss, being a stakeholder in the work being done, was really asking those two questions from Part 1 so he could evaluate the current contract and plan for new ones. I'd tell him, "I'm doing work, but I have no idea where it's going and when we'll get there!" It was a software process that was missing the point.

The same questions that the stakeholders are asking are the same questions to use to measure one's software process. If you have an answer, the process is working, and if it takes more than 2 minutes generate or explain the answers, there is a problem.

Where is the software going?

Every member of the team should know the answer to this question. If one is a Team Lead or Product Manager, the answer needs to go both ways. It goes up to the stakeholders, like Business Dev, Marketing and the C-Suite. If they are not directly planning what the software must be capable of, they at least need to be made aware what is coming next in terms of features and improvements. 

This question must flow down to the devs as well. If they do not know where the software is going,  motivation drops and future features are not planned for. Keeping an air-gap between business and dev stifles ideas from those that know the technology the best, and lowers the meaning of work they are doing to simply "doing what they're told."

When looking at your software process, each member on the team must be able to know, quickly and easily, where the software is going. 

• Can anyone look at the list/board/spreadsheet/diagram and know what is coming next? 
• Can they see where their work fits in right now? 
• Is there a place to write down new ideas? 
• Are the milestones front and center? 
• Can the reason of the feature (or entire project) be summed up in two sentences that give meaning to the devs and makes sense to the business side? 
• What will come next, after this cycle of development?

Use your process to understand where your software is going.

When will it be ready? 

The schedule is vital to a business, so the development team cannot simply stall for time, or make excuses like, "it's ready when it's ready." Demos, marketing, trade shows, and everyone's paycheck is resting on these dates. Any slips need to be communicated. (Any early finishes should be celebrated!) 

A memorable quote by Walt Disney, which I like to repeat, is "Everyone needs deadlines." Walt goes on, but that's the important part. The software schedule sets everyones expectations of "done". And without it there is confusion, miscommunications and unmet expectations. Developers need to fit their work within the timeframe, and cannot go off the path just "because" ("because this new way looks cooler", "because I want to refactor now"). Estimation is an important skill for any team, and fitting what must be done within time bounds is part of the planning process, not the middle or end of the execution phase.

The schedule has to be clear, and your process must make it clear. Having business side stay out of your hair until the software is supposed to be done, and then breaking down the door after a missed date is terrible. Don't work like that! Use the process to quickly and easily inform the business side and guide the development side. 

• When is this cycle of software development going to be done?
• How close is the team to being done? How much more time is needed? 
• When is the code frozen (if not continuous)? When will QA look at it? When will customer see it?
• Is the team currently on track? How accurate were previous estimations? 
• When is the next major milestone?

Use your process to understand when the software will be ready. 

If You're Happy and You Know It

What is the ultimate measure of your software process? Whether or not the stakeholders are happy! That's it. If they are unhappy, look in to ways to improve the process to better answer the two questions. (Hint: this likely involves bringing them into your process, not pushing them away). If they were happy, then great! And take a little time to see how to make your life easier.

Stakeholders looking at your software might not be very technical. Metrics and procedures mean very little to them. When they evaluate they are looking at two things:

1. Did you do what you said you were going to do?
2. Was it ready when you said it was going to be ready? 

Be able to answer "yes" to those two questions and you will have some happy stakeholders.

But you knew that was coming, right? Because your software process, quickly and easily, has been answering those two questions all along, so everyone knew exactly where the software was going and when it was going to be ready.

That's the whole point of all this process, and that's how you know it worked.

The Point of Process - Part 1

A software development process was very real during my time in the defense industry, but basically not taught during my time in college. I could see mixed views from the open range of small businesses and development shops, depending where one worked. There is plenty of material out there too! Manifestos, guidelines, workshops and so on, all geared to learning and following a particular software process. And all this truly has made software development better, and allowed for better software.

If one did not have varied experiences, or there was not time to do extra reading, or was just getting starting in software, they might wonder, "what is the point of all this process?" Or if one was getting frustrated with a software process, because it was too much pointless work or the process never told them anything useful, they might wonder, "what is the point of all this process?" And some people just need to be reminded, because the point of a software process is not the process itself.

Software Process Answers Two Questions

Where is the software going?
When will it be ready?

That's it. Be it waterfall or agile or something else, it all boils down to these two questions.

Where is the software going? 

Software needs a goal. The goal could be planned months ahead or days ahead but there needs to be an end state. That could be a milestone with a defined set of features or requirements, or a rolling set of features as business needs change, and the process keeps everyone involved on track. Each person can state why they are doing the things they are doing (writing code, performing tests, etc) and, to a higher level, where those things fit into the entire software system.

When will it be ready?

Schedule is important. The software process tells everyone when something must be done. This helps the business guys plan sales demos and constrains the developers to stop tinkering. If the team cannot state when the software will be ready, chances are it will not be.

Who Asks the Questions

The process is for the software and the software is for the stakeholders. Only minimally is a stakeholder the developer. Sometimes it could be Tech Lead, Scrum Master, Program Manager or CTO. Indirectly it is the end-user. The stakeholder should be the business side of the house, and very often it's the person writing the check. The stakeholders are the ones asking the questions, and the answers to those two questions (from the dev team) is sometimes their only view into the software process.

Development is not the point. In the end, process is not the point either. Keep your stakeholders happy and meet their expectations, which can be done by answering two simple questions. You might use a software process to help answer those questions, too : )

Meaningful Software Tests

Software Testing is vital to any project, though not all software tests are meaningful. Not a big deal if a few extra automated tests get written or run. Debilitating if the tests cannot catch what they should, by forcing errors to be found outside the dev team or becoming a maintenance nightmare. So what tests should you write?

Testing Types

There exist about three types of software tests:

• Unit Test - the lowest level of test. Against a single "unit" of code (a class, a group of methods, an algorithm, etc). One unit is tested in isolation and any other units are mocked to keep functionality going and maintain control of the test.
• Integration Test - Multiple units tested together, but still the same software system. Can have dependencies outside the software, like a database or even, gasp, an internet connection. Much less is mocked at this level. 
• End-To-End Test - the whole deal. Testing from one end of the system (user interface) to the other (database, algorithms) and back. Runs real code in real environment with (custom) test runners. Nothing is mocked - it doesn't need to be!

People might use different terms for these types of tests, and different approaches might blend somewhere in the middle of these. Want a long list to amaze your friends,mor give you some good ideas? Here.

But honestly, the type of testing you do isn't the most important thing. That you have testing is important, but varying opinions and success stories will advocate for one type or another.

Is There Meaning In These Tests

The type of test is not the issue, rather it is "are these tests meaningful?" Each test that is written should have some meaning behind it - test out a particular function or flow so you know it works. Test out a particular business need or requirement so you know its covered. Test a particular corner case so you know that bug will not come up again.

You need to be confident in your testing. These are little bits of automated software that will prevent errors down the line (where they are much more expensive to fix). When you run your tests, you need to know the functionality they tested and be confident that functionality was performed correctly.

Two quick examples from the past.

Back in the defense industry, documentation was king. Requirement number to test number to test results. Who wrote 'em, who performed 'em, who witnessed 'em and did they each test pass or fail. If you ever wondered how complex programs with hundreds or tens of thousands of requirements can get approved, its because of the documentation backing it up. Each test was performed for a specific purpose, and that purpose was written down. Anyone looking at the resulting documents would know what was being proved when some test step was being performed.

A second example was a past contract. They had pure unit tests that were never to turn into integration tests. One test for one unit. Code coverage was measured and the number of tests were counted. If you were to ask why we were writing all those tests, it was to get code coverage up and have all the code tested! The bad part was that every piece of code was tested in a vacuum (no database, no outside services, minimal component communication), so when the software verification group looked at something (or worse, they deployed!), lots of things were found because it was the first time those pieces were mingled outside of developer testing. There should not have been confidence that those unit tests were a predictor of safe software.

It Depends

Like many software answers, "It Depends." The type of testing you use and how much you test will be dependent on your goals and business needs. Make sure your code has tests, that those tests have meaning, and your teams understands why the tests are there.

Don't waste time writing tests just to fill metrics. If 100% code coverage is important to, or required in, the project, write those tests! But if you made up a number for coverage and then force yourself to reach it (or even better, just keep adjusting your target number!), you are wasting time. Don't test code just to check off some software engineering list - the number of tests you run is as meaningful as SLOC metrics. Tests need to be updated and maintained, forever, and are written with each new feature. If time is being wasted, find out sooner rather than later!

Meaningful tests will help the team too. Later work load will be reduced by catching real bugs early, avoiding those crazy weekend or late night sessions to "GET IT FIXED!!!1", not to mention freeing up your Verification or QA groups a bit. The dev team will have confidence and faster feedback that what they write will work, or not, boosting productivity. Continuous Integration is not possible without tests you can trust. Writing tests that have meaning makes test writing valuable, leading devs to avoid half-backed work (and insidious false negatives).

One final tip. Tests will have more meaning the closer you can get to real code on real hardware. This is also a more expensive environment to set up, and maybe cannot be automated, so it is another decision to be made. Where you are able, get as close as you can to real code on a real system.

 Happy Testing.

NodeJS on EC2 Instance

I needed a clean server to run a quick NodeJS server. We have been using Node for a while now and I haven't set up a server from scratch for a while. Either it's been working and we've just been using AMI images, or I used Elastic Beanstalk, which has done the Node setup already. There are some good blogs or tutorials on how to build node from scratch, but all the building is kind of tedious, especially if you picked a cheap server without all the processing power of the more expensive tiers.

So, to do this faster,

sudo yum update
sudo yum install openssl openssl-devel
sudo yum groupinstall "Development Tools"

That gets a few things ready on the system, and yum is quick. 

Next, I'm using Node Version Manager, or NVM for short. This is just a glorified bash script, though I use it on my dev machine. It's really great. (Especially if your last contract was still using Node 0.10 but pet projects want to use the latest stable version).  Also, it avoids all the source code building, which is the main point of this blog post : )

From the readme (all one line):

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.0/install.sh | bash

And that's it! I exited my ssh session, and came back. Then quickly ran

nvm install 5.9.1

Which worked flawlessly, and I could run node and npm in my terminal. 

$ node -v
v5.9.1
$ npm -v
3.7.3

I personally found that quicker than the other blogs out there. Hopefully it saves you some time as well.

Later Edit: When running after this install, I had an issue where my installed user could find Node (like above) but other users could not. The option I went with was from this blog, running the command:

n=$(which node);n=${n%/bin/node}; chmod -R 755 $n/bin/*; sudo cp -r $n/{bin,lib,share} /usr/local

And to quote the blog:

The above command is a bit complicated, but all it's doing is copying whatever version of node you have active via nvm into the /usr/local/ directory (where user installed global files should live on a linux VPS) and setting the permissions so that all users can access them.

This command would need to be run each time I ran a version. I also had to make sure /usr/local was in the $PATH of the users that needed Node.  And alternative to these extra commands would be to just use the nvm-global version of nvm (a different repo), which I might try next time.  But for now, I'm set up so will get back to it!

Better Faster and Cheaper Expectations

A common theme is software development is the old adage of "Better Faster Cheaper." Which includes everyone involved in the project, whether they know it or not. We all, especially devs, want our software to be better (less bugs, more features, faster performance, painless DevOps). We all, especially managers, want our software to be done faster (because a business/demo schedule is in place). And we all, especially those with the checkbooks, want our software to be done cheaper (in terms of expense, and software is not cheap!). 

As the adage goes, "you can only pick two." The "best" place in the triangle is probably the center, but that's just equalized. Maybe less surprises and less risk, but everyone is out to optimize something and the focal point shifts for a number of reasons. In some cases, it has to! (or at least should.)

So, where is your project aiming?

Better and Faster, but not Cheaper

Great software delivered quickly. Everyone likes great software that does what it is supposed to do without problems. And there are good reasons to need it fast, like being first-to-market, or iterating features faster than your competition, or your investors are getting antsy. It can be done!

And, it comes at a price. For any part of the project that can be developed in parallel, you can throw people at the problem. Maybe some incentives will keep the team working harder. Buy the good machines and latest tools. Maybe you can hire a real rockstar team of experienced devs, or just buy the technology and services your system is missing.

Whatever the approach, the need for better and faster is outweighing the cost constraints . . . and that's the point! If all those reasons to need it fast are right, and the software is great, you have a winner on your hands and would gladly spend that cost all over again.

Better and Cheaper, but not Faster

Great software at an economical price. Everyone likes great software that does what it is supposed to do without problems. And there are good reasons to need to done cheaper, like lack of investment, changing company profit margins or shrinking development resources.

Without resources, time slows down. Be it lack of motivation or reduced effort, the software can still be good but do not expect it to be done quickly. I see this in a lot of Open Source projects which put out great software, but most of those devs are one or two people who have other jobs and interests. They build and maintain great software, for free (can't get cheaper than that!), but sometimes their Issues list is long or v2.0 has been living in beta-mode for a while.

Progress won't happen quickly . . . and that's the point! Resources are being assigned elsewhere (or never existed in the first place), and the project is still moving forward without burdening cost (time and talent).

Faster and Cheaper, but not Better

Everyone likes great software that does what it is supposed to do without problems. And they want it now! For free! This area of the triangle is what causes developers the most strain. They want to spend more time on that problem, follow good development practices, investigate all the alternatives, re-write that ugly piece of code from last month, etc. - but deadlines and budgets don't allow for it. During my time in the defense industry, I was reminded that I wasn't creating the absolute best software for my task . . . I was fulfilling the stated requirements with the contract constraints that were in place. Talk about strain!

While customers and managers are always monitoring for faster and cheaper, they also want better! This is where a dev shop can fall into problems, by promising better without any means of faster (except "work weekends forever") or eating the cheaper themselves. Having someone "on the dev team's side" that can explain this situation is vital to the health of your team and project.

Faster and cheaper are important and often inflexible . . . and that's the point. When those are the biggest priorities, all parties should be made aware just what part of better is going into Phase 2. There are some times where faster and cheaper make a lot of sense as well, like internal prototyping or feature proof-of-concepts. Better can be handled later, when someone else is paying for it!

Expect All Three

Of course, anyone involved with a software project will say, "I want it all!!!" And they're right, they do want it all. They should have it all too! But the grumpy software veteran will say "nope, you can only pick two" and the uninformed customer will be surprised when the target starts drifting towards one corner of the triangle, only to want to focus on a different corner the next week. The unmonitored devs will push to better every time, especially if they're allowed to just refactor it all again, only to wonder why their schedule is shot and they're rushing at the end. Can all three really be done?

There is no mystical triangle to guide our software, but all decisions will have some outcome. If everyone is expecting the same thing, and the team performs, everyone is happy and not thinking if it could have been better, or faster or cheaper. If, however, expectations were not equal, then someone is left wanting more and pointing fingers at the imagined target on the triangle. "It should have been faster." "What about feature X?"

Yes, we can have all three . . . in a way. If expectations are set correctly and maintained throughout the project, everyone is getting what they expect to get and don't think about the triangle . . . and that's the point! With a defined and limited set of requirements that are perceived as "better", the team can work quickly and cheaply. With a defined budget, that "looks economical" and isn't skimmed, for the right team, good software can be created quickly. With achievable milestones that are "fast enough", good software can be created cheaply.

The triangle is useful to explain compromises while setting expectations. It is not a compass to magically redirect software efforts, nor a map that explains "how we got here is your fault." Set clear goals, keep communication lines opens and enjoy great software that is so much better, faster and cheaper that no one even thinks about it : )

Card Game Project - Deployment

With some working components, and before jumping into the game logic, I want to try out building a native version for at least Android.

Cordova

Building with Cordova worked pretty well. I spent more time installing packages than I did building the app. The best part was the app works and looks great on the device. Special thanks to this blog for documenting their steps as well.

npm install -g cordova
cordova -v

Installing with npm is a breeze. I am using Cordova 6.0.0.

cordova create polycardwarapp com.chimmelb.polycardwar PolyCardWar

This initialized my application in the polycardwarapp directory.

cd polycardwarapp/
cordova platform add android

Build my dist/ directory and copy it to the /www directory of the app.

cd ..
cp -R dist/ polycardwarapp/www/
cd polycardwarapp/
cordova build android

This is where I got stuck. I add the Android SDK tools installed but either never used them, or needed to update them. 

android update sdk -u -a

That command made me agree to many EULAs, and started updating some build tools. I waiting while updating 24, 23, 22, 21 and quit soon after that. Then the build worked flawlessly.

cordova emulate android

Emulator was OK, and installed the apk created at platforms/android/build/outputs/apk/android-debug.apk on my phone. Started up quickly and was responsive. 

PhoneGap

The instructions for PhoneGap building were pretty straightforward, and was more app-based than command line. I followed their instructions for the sample app, copied my dist/ directory into the www/ folder, and the next build worked! 

I was suprised to read this: 

Apache Cordova is the engine that powers Adobe PhoneGap™, similar to how WebKit powers Chrome or Safari. 

...so perhaps that's why these were two positive experiences.

Mobile Chrome App

My first try was the documentation to build a chrome app, included in Polymer. This worked enough to build an app, but the routing in the app didn't work for me and kept getting toasts about not finding certain app paths.

Next Steps

I'm sure Cordova and PhoneGap have many more options, like the icon for the app, portrait-only view models and other cool things. I sometimes get an app routing error when first starting the game, and there could be other errors hiding about that will only be visible in an app format, rather than on the web browser (PhoneGap's tooling would probably be more useful in that case too). The Crosswalk project is also an option (or plugin) that could be useful.

But deployment is possible, and that's what I intended to find out. The game needs a lot more logic, but the layout and user input ("clicking") all worked on a mobile device in an app format. A more formal solution would be needed before hitting an app store, but the biggest risk of "Is this even possible?" has been cleared, so we can continue back to the app!

Tracking

This was another day, though there were some other project hurdles to handle today. And like I eluded earlier, installing android packages was a bigger time-sink than anything else. The actual build tools worked great! 

Card Game Project - Components

There are two components that are custom to this game, the <game-card> and the <card-stack>, which is a collection of <game-card>s.

Cards

The simpler of the two will be the game card. Like your standard playing card, it need a front and a back. The back will have a logo (the Polymer logo), the same on all cards. The front will have text that reflects some rank (in this game, it's basically 1-6 plus 3 special card types).

The first trick is to not show both sides of the card at the same time. I took out the other styling attributes for the css, and just wanted to show the affect of a relatively positioned outer <div> (so it's still affected by flexbox correctly), and two absolutely positioned inner <div>, with classes front and back. The outer div defines the css transition, and that the back of a flipped card can be seen (backface-visibility: hidden;). The front of the card is flipped over with  transform: rotateY( 180deg), and the front and back are both rotated 180 degrees in the same direction to get the affect.

That flipped class is applied with Polymer's special attribute data binding, class$="front {{isFlipped(show)}}", which binds the calculation of the class to a function in game-card's script called "isFlipped", that is evaluated every time the value of the attribute "show" changes (internally to game-card, or applied to the element from elsewhere). When the class appears or disappears, the css animation is applied.

For now, an "on-click" event is present to flip the cards as needed.

Stacks

Cards in this game are played from stacks (or piles). Two things I wanted to do for stacks was

1. Proportion the cards dimensions
2. Fan the cards so I could see them all from top to bottom.

My first step was to make cards that had a respectable shape! Normal playing cards are 64mm x 89mm so we want to keep that proportion. Plus, in the card stacks I want to "fan out" a stack of up to 5 card and still fit in the width of the <card-stack> element (which is always resized with flexbox). A little calculation is needed, and the solution I've used was this (polymer properties removed):


With Polymer, getting the size of elements (or their parents) is tricky if not done at the right time where all the values are 0. Here I used the IronResizeableBehavior to catch an iron-resize event, and if the height is > 0 (meaning it actually rendered), performed some calculations to preserve the aspect ratio. 

Fanning the cards needs to happen from inside the <card-stack>, but outside of <game-card>, so the aspects and behaviors of the cards are independent of the styles applied. Using Polymer's <dom-repeat> template, we can list the cards in an array, and with CSS just stack them on top of each other. This style was very calculated, so this was done with a function bound to three attributes of the <card-stack> (so will be recalculated each time).

Tracking

This step is wrapped up, so I created a branch. https://github.com/chimmelb/polycardwar/tree/step-components

The resizing of cards (and where to bind values in the layout) took about 3 hours, the card flip was about 2, and the fanning elements was about 1. Measuring time is funny, because it really was about a day between meetings, lunch, the other projects that needs attention, etc . . . cramming 6 hours of good work into 9 hours of my day : )