Software Development Methodologies – Agile

Although the waterfall method of software development has many benefits, it also receives its fair share of criticism:

• There is a cost to not getting things perfect up front. Once the design phase begins, changes are difficult to get into the project. What if you realize that the requirements were missing a major element once the project is well into the build phase or testing?
• The system requirements, although compiled from the input of many team members, can feel like they are being captured in a vacuum. They don’t have the benefit of being fine tuned against a demo version of the system.
• Using a word processor to capture the essence of an idea or vision can be frustrating and is often limited by the effectiveness of the analyst doing the writing.
• Although the phases of the process create clear delineations between roles, they can also create barriers between collaboration.

Wouldn’t it be great if we could build software in a more iterative fashion? What if we could build small pieces of the overall system, show those pieces to the team, get feedback, and then tweak the direction of the project?

That is exactly what agile methodologies set out to accomplish.

There are many flavors of agile development, but the process typically looks something like this:

• The system is built in a series of iterations called sprints. Sprints last a pre-determined period of time–usually two to four weeks.
• The requirements are expressed as short user stories which are written and prioritized by the business stakeholders. Stories are placed into the project backlog.
• The team meets to plan a sprint by estimating how long the stories in the backlog will take to build. Then, the team selects which stories will be built in the upcoming sprint and commits to them. Once a sprint starts, it cannot be altered. The team meets briefly every day to review progress and overcome roadblocks.
• At the end of the sprint, a demo of the system is presented to the team where everyone can voice feedback.
• Then, the whole process starts over again with a new sprint.

When done correctly, an agile process focuses the efforts of the team and encourages adaptation to better meet changing requirements. Instead of discouraging change, it expects and embraces it.

Software Development Methodologies – Waterfall

In my last post, I talked about different methods of writing a novel.  In software, things are much fancier.  Instead of methods, we have methodologies.  Also, you don’t “use” things–you “utilize” them.  Example: “I utilized the appropriate methodology to garner the appropriate resources to achieve my deliverables.”

All kidding aside, when most companies (especially larger companies) write software systems, they have a methodology that they follow.  Large software projects can involve dozens of stakeholders, project managers, various analysts, testers, several development teams, and other sundry players.  Things tend to go more smoothly when the project is organized in a such a way that people recognize the framework and know how their role fits into the big picture.

Another point of software methodologies is that they break the software system into its component parts.  For instance, you don’t start writing code on day 1 of a software project.  And you don’t start testing on day two.  There’s an order to things.

The most established type of software development methodology is called The Waterfall Method.  The waterfall method involves starting at the beginning and working through the project in a linear fashion.  In the purest form of the waterfall method, once you move into a new phase, there is no going back–there is only moving forward (thus, the image of moving down a series of cascading waterfalls).

The phases in a waterfall approach typically look like this:

1) Ideation.  This is sometimes a formal step and sometimes is an implied one.  But, this is basically the moment when someone or a group of someones brainstorms some ideas and comes up with a novel approach as a solution.

Deliverable at the end of this phase: High-level scope document

2) Requirements.  This phase involves the owner of the system (usually someone outside of the IT department) and one or more business analysts sitting down and writing a document that lays out what the system needs to do in detail.  The resulting document is usually about two inches thick and gets reviewed by every member of the team on day-long, soul-sucking conference calls.  When everybody agrees on the requirements of the system, this phase is complete.

Deliverable at the end of this phase: Detailed requirement document

3) Design.  Enter the techies.  This is where technical acumen is set loose on business need.  The developers and system analysts read the aforementioned requirements document, digest it, cogitate on it, and then create a technical design that meets the requirements of the system.  For instance, you might hear a computer programmer say, “Hey, customers are going to have to interact with this system.  Should we build a website where customers can login?  Or should we create a smart phone app that customers can download?  Or should we send out CDs like AOL used to do circa 1996?”

Deliverable at the end of this phase: Usually some diagrams, data mappings, and a detailed estimate of how long the system will take to build

4) Build/Integration.  Once the techies have worked out their plan of attack, they start coding.  If the project involved more than one of the company’s systems (like the accounting system, the HR system, the main website, etc.), there is probably some integration work to make sure all the systems play nice together.

Delivered at the end of this phase: a working, version of the system in a test environment

5) Testing/Quality Control.  A group of quality control analysts takes the requirements document and the working (or semi-working) version of the system and begins to test the system to make sure it actually does what it says it does.  When the testers find something that is broken, they report the defect back to the developers who in turn fix it.

Delivered at the end of this phase: a tested, (hopefully) working version of the system (still) in a test environment

6) Implementation.  The system has to be rolled out to the production environment (aka, the “real” version that the “real” users of the system can get to).  Sometimes this is done all at once, but sometimes there is a “beta” version released first.  Sometimes this involves getting your app into the Apple store.  In 1996, this involved making CDs of your software that you will send to your customer via snail mail.

Delivered at the end of this phase: the system is launched out into the real world!  Yea!!!!

7) Maintenance/Support.  This phase involves the monitoring, support, and operation of the system.  ‘Nough said.

Delivered at the end of this phase: things don’t fall apart; the lights stay on; the wheels stay on the bus; <insert metaphor here>

So, there you go.  That’s a waterfall approach to creating a software system.  Is it boring?  Yeah, a little bit.  Is it obvious?  Pretty much.  Is it helpful in the real world?  Absolutely.

What’s so great about it?  It splits up the creation of the project into defined phases with defined roles and defined deliverables.  It brings structure to chaos.

Wikipedia’s take on the phases of the waterfall method

Could such a process have value in the creation of other things?  Like the writing of a book.  What an astute question!  Stay tuned for the answer….

Novel Writing Methods

So, you want to write a novel? Cool.

There have been countless book written in the history of humankind. How hard can it be?

Go down to your local bookstore, go to the writing reference aisle and pick up the book you find there. What’s that you say? There are 300 books in that section, and they all advocate a different way of doing it?!? Hmmm….

Maybe we should check with an expert. Have any famous novelists written revealed their top-secret novel-writing methods? Sure. Lots of them have, and the methods they use are all over the map.

I’ve researched writing methods quite a bit, and in my opinion they all basically fall into one of the following four categories:

1) Outline until you can’t outline anymore. Got an idea for a novel? Cool. You probably know how it starts, and you probably know generally how it ends. Take that seed of an idea, break it into three parts: a beginning, a middle, and an end. Now, take those three parts, and break them down further. Keep breaking things down until you have a comprehensive scene list. Make sure you have an inciting event. Make sure you have a point of no return. Make sure you have a rise from the ashes. Now, take each your scenes and detail them out: what characters are in the scene, what is the setting, what is the scene’s emotional arc, etc. By the time you’re done, you should have an outline large enough to choke a pack of angry termites. From there, it’s easy. You just write your scenes.

Arguments for this method: A novel needs structure. There are formulas for writing books. Use them. Why get 30 chapters into a novel and realize, “Man, this idea sucks”?

2) Just write. Got an idea for a novel? Cool. Get out your notebook and your favorite pen, and let the magic start. Skip the outline, skip the structure. Turn off your left brain, and just let your creative, intuitive, emotional self guide you into a brilliant story.

Arguments for this method: If you as the author know what’s going to happen in your book ahead of time, when you write it, it will be predictable and feel scripted. However, if you don’t know what’s going to happen in a book, the book will have to feel spontaneous–because it was! After all…aren’t writers supposed to write?

3) Let your characters write your story for you. Got an idea for a novel? Cool. That idea probably involves a main character, maybe a sidekick, maybe a cool villain, maybe a love interest. Take each of those characters and explore them thoroughly. Interview them. Figure out their motivations. Give them depth. Make them multi-dimensional. Then, when you know your characters inside and out, when you’re dreaming about them, when they feel like your best friends, put them into a scene together and see what happens. The words will fly off the page. Your characters will *know* whether a scene feels right or not, and they will tell you.

Arguments for this method: Do you remember what happens in Gone with the Wind? Probably not. Do you remember Scarlett O’Hara and Rhett Butler? You know you do. Great characters make great novels.

4) It doesn’t matter how you write it–just power through. Got an idea for a novel? Cool. Your job as an author is to write a sh***y first draft of that idea. Once you have written it, you can always revise later. You can get an agent who will suggest changes based on their in-depth understanding of the industry. You can hire a copy editor to fix all your bad grammar. You can hire a story editor to help you cut out those twenty-page expositional sections that seemed so important when you penned them. You can submit your work to your writing group, and they can gently urge you to remove the Alice-in-Wonderland-esque dream sequence that you thought worked so well as the opening prologue.

Arguments for this method: The worst rough draft that has been put down on paper is better than the best idea that is still floating around in your head.

Got your method figured out? Yeah, me neither….

Here’s the Idea….

I have been a software developer for the past fifteen years.  Software suits me.  It appeals to my logical, left brain, and I’m sure I’ll pursue it for many years to come.

A few years back, a friend of mine introduced me to author Nelson DeMille.  I picked up a DeMille novel and took it for a spin.  The witty, first-person point-of-view, unique mix of humor and suspense, and engaging dialogue captured my attention and never let go.  Soon, I had finished his whole back catalog and moved on to other mystery/suspense writers such as Lee Child, Michael Connelly, and Barry Eisler.

As I started to embrace my new identity as mystery novel junkie,the thought occurred to me that I should consider writing a novel.  I would love to say that this inspiration hit me because I was moved by particularly brilliant piece of prose.  But, the fact is that the thought popped into my mind when I was trying out a new audiobook from the library that was truly terrible.  The characters were flat, the plotlines banal, and the literary devices broken.  I couldn’t finish the book, but I remember thinking: “This book is horrible, but it got published.  Not only did it get published, but it was popular enough to be put out as an audiobook and picked up by my local library.  If this guy can get published, how hard can it be?”

I sharpened up a pencil and over the next few months, I tried my hand at writing a novel.  I got 10 pages into about 3 different ideas.  Eventually, I took a workshop at a local writing school, and I made a lot more progress.  However, I eventually found myself stuck and frustrated.

The main issue that was I was struggling with was my writing method.  Should I just start writing and see where the story went?  Should I create a detailed outline?  Should I create character summaries?

I recently decided to stop what I was doing, take a step back and analyze the situation.  After putting some thought into the matter, I had an idea.  Here it is:

A novel and a software system have a lot of things in common.  They both need clearly-defined requirements.  They both involve planning and design to ensure consistency and usability.  The finished product is built through the employment of artfulness and skill.  I know how to build software systems.  Why can’t I apply the same processes to create a novel?

So, there it is.  I’m going to give it a go, and I plan on using this blog to chronicle the experience.