The great git workflow discussion

It's been 4.5 years now since Vincent Driessen published his thought-provoking article on git branching workflows: A successful Git branching model.

In a slight conflation of names, Driessen's workflow model has been known by the name of a toolset that he also contributed, which helps implement the model: git-flow.

In the years since, there have been a wealth of variations, elaborations, and alternatives tossed into the ring. Reading them is a fascinating way to keep up with the ongoing debate about how teams work, and about how their tools can help them work.

• A successful Git branching model

  We consider origin/master to be the main branch where the source code of HEAD always reflects a production-ready state.

  We consider origin/develop to be the main branch where the source code of HEAD always reflects a state with the latest delivered development changes for the next release. Some would call this the “integration branch”. This is where any automatic nightly builds are built from.

  When the source code in the develop branch reaches a stable point and is ready to be released, all of the changes should be merged back into master somehow and then tagged with a release number. How this is done in detail will be discussed further on.

  Therefore, each time when changes are merged back into master, this is a new production release by definition. We tend to be very strict at this, so that theoretically, we could use a Git hook script to automatically build and roll-out our software to our production servers everytime there was a commit on master.

• Why Aren't You Using git-flow?

  I’m astounded that some people never heard of it before, so in this article I’ll try to tell you why it can make you happy and cheerful all day.

• git-flow Cheatsheet

  Git-flow is a merge based solution. It doesn't rebase feature branches.

• Issues with git-flow

  At GitHub, we do not use git-flow. We use, and always have used, a much simpler Git workflow.

  Its simplicity gives it a number of advantages. One is that it’s easy for people to understand, which means they can pick it up quickly and they rarely if ever mess it up or have to undo steps they did wrong. Another is that we don’t need a wrapper script to help enforce it or follow it, so using GUIs and such are not a problem.

• On DVCS, continuous integration, and feature branches

  The larger point I’m trying to make is this. One of the most important practices that enables early and continuous delivery of valuable software is making sure that your system is always working. The best way for developers to contribute to this goal is by ensuring they minimize the risk that any given change they make to the system will break it. This is achieved by keeping changes small, continuously integrating them into mainline, and making sure there is a comprehensive suite of automated tests to verify that changes behave as expected and don’t introduce any regressions.

• My Current Java Workflow

  I then setup a Jenkins job called “module-example snapshot”. This checks out any pushes to the develop branch, runs the gradle build task on it (which runs tests and produces artifacts on successful test passes) and then pushes a snapshot release to our in house artifactory server. This means any push to develop will trigger a build that releases a snapshot jar of that module that others could use for their development.

• GitFlow and Continuous Integration

  So, what does one do with this information? Is use of GitFlow or promiscuous integration a bad idea? I think that it can work very well for some teams and could be very dangerous in others. In general, I like it when the VCS stays out of the way and the team gets in the habit of pushing changes and looking to the CI server validation that everything is ok. Introducing promiscuous integration could interrupt this cycle and allow code changes to circumvention the mainline longer than they should. This branching scheme feels complex, even with the addition of GitFlow.

• Another Git branching model

  But cheap merging is not enough, you also need to be able to easily pick what to merge. And with Git Flow it’s not easy to remove a feature from a release branch once it’s there. Because a feature branch is started from develop it is bound by its parents commits to other features not yet in production. As a result, if you merge a feature without rebasing you always get more commits than wanted.

• Branch-per-Feature

  Most of this way of working started from the excellent post called “A Successful Git Branching Model”. The important addition to this process is the idea that you start all features in an iteration from a common point. This would be what you released for the last one. This drives home the granular, atomic, flexible nature that features must exhibit for us to deliver to business in the most effective way. Git flow allows commits to be done on dev branches. This workflow does not allow that.

• git bugfix branches: choose the root wisely

  The solution I like best involves first finding the commit that introduced the bug, and then branching from there. A command that is invaluable for this is git blame. It is so useful, I would recommend learning it right after commit, checkout, and merge. The resulting repository now looks like Figure 3, where we have found the source of the bug deep inside the development history.

• Some thoughts on continuous integration and branching management with git

  Given a stable/production branch P, and a set of feature branches, say, FB1, FB2 and FB3, I want a system that:

  • Combines (merge) P with every branch and test it, say P + FB1, P + FB2, P + FB3.
  • Select the successful branches and try to merge them together. Say FB2 failed, so we would try to build and test P + FB1 + FB3 and make it a release candidate.
  • Should a conflict appear, notify the developers so they can fix it.
  • The conflict resolution is saved so not to happen again.
  • The process is repeated continuously.

• A (Simpler) Successful Git Branching Model

  At my work, we have been using a Git branching strategy based on Vincent Driessen’s successful Git branching model. Over all, the strategy that Vincent proposes is very good and may work perfectly out of the box for many cases. However, since starting to use it I have noticed a few problems as time goes on

• Two Git Branching Models

  In current projects, we tend to float between two branching models depending on the requirements of the customer / project and the planned deployment process.

• Git Branching Model

  A workflow for contributions is usually based on topic branches. Instead of committing to the particular version branch directly, a separate branch is made for a particular feature or bugfix where that change can be developed in isolation. When ready, that topic branch is then merged into the version branch.

• What is Your Branching Model?

  Perforce from the middle 90’s and Subversion from 2001 promoted a trunk model, although neither preclude other branching models. Google have the world biggest Trunk-Based-Development setup, although some teams there are going to say they are closer to Continuous Deployment (below). Facebook are here too.

• Git Tutorials: Git Workflows

  The array of possible workflows can make it hard to know where to begin when implementing Git in the workplace. This page provides a starting point by surveying the most common Git workflows for enterprise teams.

  As you read through, remember that these workflows are designed to be guidelines rather than concrete rules. We want to show you what’s possible, so you can mix and match aspects from different workflows to suit your individual needs.

• Git Branching - Branching Workflows

  Now that you have the basics of branching and merging down, what can or should you do with them? In this section, we’ll cover some common workflows that this lightweight branching makes possible, so you can decide if you would like to incorporate it into your own development cycle.

SQLite and Derby

I was interested to stumble across the web document: The Design Of SQLite4.

SQLite4 is an alternative, not a replacement, for SQLite3. SQLite3 is not going away. SQLite3 and SQLite4 will be supported in parallel. The SQLite3 legacy will not be abandoned. SQLite3 will continue to be maintained and improved. But designers of new systems will now have the option to select SQLite4 instead of SQLite3 if desired.

SQLite4 strives to keep the best features of SQLite3 while addressing issues with SQLite3 that can not be fixed without breaking compatibility.

It surprised me to learn that "internally, SQLite3 simply treats that PRIMARY KEY as a UNIQUE constraint. The actual key used for storage in SQLite is the rowid associated with each row." I think it is good that SQLite 4 will amend that choice, and treat PRIMARY KEY more as a DBA would expect it to behave.

I like the fact that, overall, SQLite is continuing to move toward a more standard and "correct" implementation, by doing things like requiring that PRIMARY KEY columns be non-null, and turning foreign key constraints on by default.

Overall, it looks like SQLite is heading in a good direction and I'm pleased to hear that.

The overall goals of SQLite are very similar to Derby, which I am considerably more familiar with.

The Derby community, too, continues to remain active. Here's the plan for the next major Derby release, which will be coming out this summer: 10.11.1 Release Summary:

• MERGE statement
• Deferrable constraints
• WHEN clause in CREATE TRIGGER
• Rolling log file
• Experimental Lucene support
• Simple case expression
• New SYSCS_UTIL.SYSCS_PEEK_AT_IDENTITY function
• Use sequence generators to implement identity columns
• add HoldForConnection ij command to match NoHoldForConnection

Although some of those features are pretty small, a few of them are large, dramatic steps forward (MERGE, CREATE TRIGGER WHEN, deferrable constraints, the Lucene integration)

In my own professional and personal life, I haven't been spending as much time with Derby recently. I no longer write code in Java for 50 hours every week, so it's hard for me to find either time or excuses to be intimately involved with Derby.

However, I try to follow along as best I can, monitoring the email lists, spending time in the Derby communities in places like Stack Overflow, and generally keeping in contact with that team, because there's a superb community of brilliant engineers working on Derby, and I don't want to lose touch with them.

So: way to go, SQLite, and way to go: Derby!

Another dry year

From Slate: The Thirsty West: Where’s the Snow? The mild California winter will exacerbate the terrible drought.

New data show that California will be starting the summer dry season with a snowpack around the lowest levels since recordkeeping began nearly a century ago. The data were collected by hand over the last week as part of California’s annual snowpack survey across the vast Sierra Nevada, an update to the automated numbers released a week ago.

Of course, record-keeping, in some sense, began quite a bit more than "nearly a century ago".

From William Brewer, via Tom Hilton's marvelous web-zine, May 30, 1864: East of Pacheco Pass

All around the house it looks desolate. Where there were green pastures when we camped here two years ago, now all is dry, dusty, bare ground. Three hundred cattle have died by the miserable water hole back of the house, where we get water to drink, and their stench pollutes the air.

Drought in California is nothing new, as Brewer's journals document.

But weather is a funny thing, as Cliff Mass notes:

Here is a notice released by the Seattle National Weather Service office:

.CLIMATE...THE RAINFALL TOTAL AT SEATTLE-TACOMA AIRPORT WAS 0.22 INCHES SUNDAY. THIS MAKES THE RAINFALL TOTAL SINCE FEBRUARY 1ST 22.87 INCHES. THIS BREAKS THE RECORD FOR THE WETTEST FEBRUARY THROUGH JULY IN SEATTLE. THE OLD RECORD WAS 22.81 INCHES SET IN 1972. FELTON/MCDONNAL

You knew this was a wet late winter/spring, particularly mid-February through mid-March. But to beat the Feb-July record in MAY is really notable.

In two months, I'm hoping to take my annual backpacking trip. We're planning to visit a lake at 10,800 feet.

I've given up on hoping that there will be snow at that altitude, though in normal years a late July visit might find 3 feet of snow there.

I am, still, hoping that there will be a lake there.

And that there won't be a repeat of last year's fire season.

So I'm going through my backpacking gear, getting it in order, being optimistic.

Stuff I'm reading, late May edition

Had a great 3 day weekend, with all my daughters in town; we took my granddaughter on a nice hike in Roy's Redwoods Preserve in Marin County.

• 1200 Feet Long, Loaded, Under Tow

  The vessel used for this exercise was CMA CGM’s Centaurus, an 11400 TEU container ship measuring 365 meters, or approximately 1,200 feet.

  The purpose of the towing demonstration was to test the capability of existing tug assets within San Francisco Bay to connect to and tow an ultra-large container vessel.

• A Short On How The Wayback Machine Stores More Pages Than Stars In The Milky Way

  Playback is accomplished by binary searching a 2-level index of pointers into the WARC data. The second level of this index is a 20TB compressed sorted list of (url, date, pointer) tuples called CDX records[2]. The first level fits in core, and is a 13GB sorted list of every 3000th entry in the CDX index, with a pointer to larger CDX block.

  Index lookup works by binary searching the first level list stored in core, then HTTP range-request loading the appropriate second-level blocks from the CDX index. Finally, web page data is loaded by range-requesting WARC data pointed to by the CDX records. Before final output, link re-writing and other transforms are applied to make playback work correctly in the browser.

• How the Neighborhoods of Manhattan Got Their Names

  For an island of only 24 square miles, Manhattan sure has a lot of neighborhoods. Many have distinct monikers that might not seem intuitive to the lay-tourist, or even to a lifelong New Yorker. Here's where the names of New York's most famous 'hoods came from.

• Slightly More Than 100 Fantastic Pieces of Journalism

  By Conor Friedersdorf

• Cisco Goes Straight To The President To Complain About The NSA Intercepting Its Hardware

  Chambers goes even further than Cisco's counsel, decrying the NSA's tactics and the damage they're doing to his company's reputation.

  “We simply cannot operate this way; our customers trust us to be able to deliver to their doorsteps products that meet the highest standards of integrity and security,” Chambers wrote. “We understand the real and significant threats that exist in this world, but we must also respect the industry’s relationship of trust with our customers.”

• Cisco's chickens come home to roost

  I wanted to point out that there's a difference between whining about how your government does something, and building a secure ecosystem.

• Queueing Mechanisms in Modern Switches

  Cell-based fabrics solve this problem by slicing the packets into smaller cells (reinventing ATM), and interleaving cells from multiple packets on a single path across the fabric.

• Troubleshooting Riverbed Steelhead WAN Optimizers

  A group of Riverbed TAC engineers have worked on an internal troubleshooting document to kick start new TAC engineers. It describes the design of the Steelhead appliance, the working of the optimization service and the setup of optimized TCP sessions, installation and operation related issues, various latency optimization related issues, on how to use the various CLI tools to troubleshoot and how you can deal with the contents of the system dump.

• Microsoft’s Most Clever Critic Is Now Building Its New Empire

  When Alchin offered him the job, Russinovich didn’t take it. But after several more years spent running his Sysinternals site–where he published a steady stream of exposés that, in his words, “pissed off” Microsoft and other tech outfits–he did join the software giant. The company made him a Microsoft Technical Fellow–one of the highest honors it can bestow–and today, he’s one of the principal architects of Microsoft Azure, the cloud computing service that’s leading the company’s push into the modern world.

The MNT roster is out!

Here's the short summary:

FORWARDS: Jozy Altidore, Clint Dempsey, Aron Johannsson and Chris Wondolowski

MIDFIELDERS: Kyle Beckerman, Alejandro Bedoya, Michael Bradley, Brad Davis, Mix Diskerud, Jermaine Jones, Graham Zusi and Julian Green.

DEFENDERS: DaMarcus Beasley, Matt Besler, John Brooks, Geoff Cameron, Timmy Chandler, Omar Gonzalez, Fabian Johnson and DeAndre Yedlin

GOALKEEPERS: Tim Howard, Brad Guzan and Nick Rimando

Here's the shorter summary:

Landon Donovan is not on the roster!

Although I think everyone expected this, after Donovan's unusual "recess" from soccer a year ago, it was still a shock. Here's what Sporting News has to say:

The U.S. has minted only one Donovan. He is the greatest field player the United States has produced; a starter at three World Cups; a player with nearly every important offensive record in his nation’s history: goals, assists, World Cup goals.

Frankly, it's hard to see our roster terrifying anyone, but Klinsmann has been a miracle worker so far, so I'm just going to hold my breath and wait to see them play.

Meanwhile, of the players that did get picked, many are very young and are unfamiliar to me. Of the players I do know, my favorites are Bradley and Dempsey, but I like watching all of them.

It's hard to believe, but the opening match is 3 weeks from today! Here it comes!

Oh, and in other news, the greatest player on the planet over the last year injured his knee today.

Shadowrun Returns: a very short review

I happened to pick up Shadowrun Returns on Steam the other day.

It was on sale, and I had been wanting it for a while (it's about a year old I think), so I jumped when I saw the sale.

It's very simple: this is totally my sort of game, and TOTALLY a fun game.

Shadowrun Returns is sort of an RPG: you have a character, and your character goes on quests, and your character develops, and you can make choices about what to do next.

Shadowrun Returns is sort of a tactical turn-based combat game: when you are in a fight, you and your enemy alterate turns, and on each turn you make a tactical decision about where to move, what cover to seek, what weapon to fire, which enemy to fire upon, etc. In that way, Shadowrun Returns kind of reminds me of XCOM.

And Shadowrun Returns is sort of an interactive novel: it's kind of a mystery story, in which the overall plot device is that of a "who-done-it", and as you go traveling around you have various conversations with characters and sub-plots and twists and turns.

That is, it's sort of a mystery story if Raymond Chandler wrote detective stories involving cyborgs, trolls, orcs, and shamanistic mages, and set it in a post-apocalyptic version of Seattle.

Yeah, yeah, that's what it is: a tactical turn-based RPG mystery novel.

Well, whatever it is, I really dig it.

If any of that sounds like it's your cup of tea, go give Shadowrun Returns a try.

Interesting things, mid-May edition

The mini heat wave is over, for now...

• Google I/O: June 25–26, 2014

  Explore the themes of design, develop and distribute during Google’s annual technology conference, as we share new product ideas and interactive experiences.

• A Riddle Wrapped In a Mystery Inside an Enigma

  Yeah, Godwin's law, whatever, whatever. My point is NOT that the NSA is the same as the Nazi party (in fact, my argument has the NSA on the opposite side). My point is that the government now treats ordinary civilians as worthy of the same sort of tactics that they once used against the Nazis.

• Vernam, Mauborgne, and Friedman: The One-Time Pad and the Index of Coincidence

  It is most likely that Vernam came up with the need for non-repetition; Mauborgne, though, apparently contributed materially to the invention of the two-tape variant. Furthermore, there is reason to suspect that he suggested the need for randomness to Vernam. However, neither Mauborgne, Herbert Yardley, nor anyone at AT&T really understood the security advantages of the true one-time tape. Col. Parker Hitt may have; William Friedman definitely did.

• Quite a good list, actually: 10 Articles Every Programmer Must Read

  Since most of these post are actually driven by practical knowledge, beginner and intermediate programmers can take a lot from it.

• I guess I need to learn the difference between "Market on Close", and "Market" orders: Stock Markets Had a Rough Second Yesterday

  All of the weird trades occurred at about the same time, just over 10 minutes before the close, and that explanation makes sense. There you are at 3:49 p.m., entering your market-on-close order to buy and sell a bunch of shares in the closing auction. You enter the number of shares you want, you go to the drop-down box on your computer system, you pick "Market on Close," your mouse slips, you actually click on "Market," and whoosh, off your order goes to be filled in the market.

• Definition of Market on Close Order

  A "market on close" order is a market order that is to be executed as close to the closing price as humanly possible.

• Trading FAQs: Order Types

  On the close

  A time-in-force limitation that can be placed on the execution of an order. This limitation requires that the order is executed as close as possible to the closing price for a security. All or any part of the order that cannot be executed at the closing price is canceled.

• Scaling Feature Flags With Zookeeper

  For those not in the know, feature flags are a way of adding a conditional to your code that lets a configurable number of users or requests through, originally designed for restricting new features to internal users before rolling them out to the rest of the userbase.

• rollout: Feature flippers.

  You might want to let a specific user into a beta test or something. If that user isn't part of an existing group, you can let them in specifically.

• Guide to Cassandra Thread Pools

  Each like task is grouped into a stage having a queue and thread pool (ScheduledThreadPoolExecutor more specifically for the Java folks). Some stages skip the messaging service and queue tasks immediately on a different stage if it exists on the same node. Each of these queues can be backed up if execution at a stage is being over run.

• Symas Lightning Memory-Mapped Database (LMDB)

  LMDB is an ultra-fast, ultra-compact key-value embedded data store developed by Symas for the OpenLDAP Project. It uses memory-mapped files, so it has the read performance of a pure in-memory database while still offering the persistence of standard disk-based databases, and is only limited to the size of the virtual address space, (it is not limited to the size of physical RAM).