Private Equity 101

Please understand: I am a software engineer, not a financier.

My understanding of Private Equity is wholly experiential.

But, after 4 months of such experience, I think I now understand enough to have some initial observations to share.

In general, in the world of business, there is buying, and selling.

Businesses buy raw materials (of which labor is a sort of raw material; thus my time is the raw material of software), fashion salable goods, and sell those goods, at a higher price than they paid for the raw materials, thus resulting in a profit.

That's not the only business model, of course, but it certainly is the most prevalent model.

Some companies are, themselves, publicly-traded. You, or I, or anyone, can buy a share of Amazon, or ExxonMobil, or Coca Cola, or General Electric. And, subsequently, you can sell your share. During the time that you held your share, if the company was successful, the value of the share increased, and so you made a profit (the company may also have paid you a dividend during that time, which complicates things slightly, but still).

The jargon for buying and selling these shares in publicly-traded companies is "equity trading," or often just "trading."

But not all companies are publicly-traded. Many are not.

If a company is privately-held, then you cannot simply buy shares of that company on the public stock exchange(s).

However, there is still buying and selling of companies in this situation.

Very large, deep-pocketed organizations, with substantial resources available to them, can arrange to buy and sell ENTIRE COMPANIES.

In principle, the idea is the same: an offer is made for the company; if it is deemed acceptable, it is accepted; the transaction occurs, and the company now has a new owner.

Since the companies being bought and sold in this manner are private, the jargon for buying and selling these companies is "private equity."

Thus: a Private Equity Firm is an investment company which makes money by buying companies, and then subsequently selling them at a higher price than it paid, realizing a profit.

Central to this approach is the idea that the Private Equity Firm has ways to increase the value of the company that it bought.

Although there are a lot of such ways, the most common approach that a Private Equity Firm uses to increase the value of a company it owns is four-fold:

1. Mergers and acquisitions
2. Price increases
3. Cost cutting
4. Leverage

Let's look at each of these in turn.

The underlying concept of mergers and acquisitions involves the notion of "synergy:" two entities may be worth more if operated as a single unit, than if operated separately.

For example, in the realm of software, if you owned a company that made a spreadsheet, and then you bought a separate company that made a word processor, and then you bought a separate company that made a project management application, and then you bought a separate company that made presentation software, you could then assemble all four of these products into a single package, which you'd call an "office suite," and you could perhaps sell that entire suite to customers for more money than you could make by selling each product individually.

Furthermore, you could sell each company's products to the customers of the other companies, so you'd end up with four times as many customers and four times as much software to sell. Yay!

Price increases are simpler. Perhaps the previous owners simply weren't charging enough for their products? In that case, raise the price, and the company will make more money. This approach is particularly popular right now in the pharmaceutical industry, but it applies to any industry, including software.

Cost cutting is also a pretty simple strategy. Perhaps the previous owners had too many staff, or were paying them overly high salaries, or were generating too many expenses. Reduce staff; cut wages; eliminate or reduce expenses.

Cost cutting can work particularly effectively with mergers and acquisitions. If you merge two companies together, you arrive at what the English so delicately term "redundancies."

You may not need two entire Human Resources departments. You may not need two entire Accounts Receivable departments. You may not need two Vice Presidents of Engineering.

Considerable cost cutting can be possible (if quite painful).

The jargon term for these sorts of things is "realizing operational efficiencies."

The fourth linchpin in the approach, leverage, is the most powerful, although simultaneously the most dangerous.

It is perhaps easiest to explain with a simple example.

Suppose that you buy the company for 100, then later sell it for 120. Well:

120 - 100 = 20

and

20 / 100 = 20%

You started with 100, and you now have 120, so you've realized a profit of 20, which is 20% of your original 100. 20% profit! That's pretty good!

But suppose that you don't actually work things this way.

Suppose that, instead, when you buy the company for 100, you actually do this by putting 15 down, and borrowing 85 from a bank, just as you might with a house or a car.

Now, when you subsequently sell the company for 120, you have to pay back your loan, and there will be some interest charged by the bank, so let's say that you have to pay the bank 90 to pay back your 85 loan. After that's done, you have 30 left, since

120 - 90 = 30

But since you only started with 15, and you now have 30, you've realized a profit of 15, which is 100% of your original 15. 100% profit! That's wonderful!

Unfortunately, leverage works just as well to amplify a loss as it does to amplify a gain.

Suppose that you buy the company for 100, then later things are tough and you can only sell it for 90. Well:

90 - 100 = -10

and

10 / 100 = 10%

You started with 100, and now you have 90, so you've realized a LOSS of 10, which is 10% of your original 100. A 10% loss. That's bad, but you still have 90 left, so you can try again somewhere else.

But if you employ leverage, well, it turns out that when you sell the company for 90, all of that 90 has to go back to the bank to pay back the original loan and its interest. You started with 15, borrowed 85, and after you repaid the loan you have 0, so you have LOST ALL OF YOUR MONEY.

And imagine that you could only sell for 89. Instead of being just a 1% change in your loss, it means that you can't pay your loan back. You're bankrupt, kaput. That's the end of your business, and of your Private Equity Firm, too.

This ability of leverage to magnify your gains is very powerful, but its unavoidable ability to magnify your losses makes it incredibly dangerous.

Leverage can turn a small improvement into a big profit, but it can turn a small mis-step into a big loss, even to bankruptcy.

This tends to make people at Private Equity Firms rather edgy, and high-strung, as they are playing for Extremely High Stakes.

Now, you'll notice that, in this portfolio of ways to increase the value of the company, nowhere did I mention

1. Make a better product

This simply isn't part of the Private Equity process, as the people who are doing these things aren't engineers: they are financiers.

In this way, Private Equity is completely and totally different from Venture Capitalism. Where Private Equity staff are almost totally uninterested in building better products, Venture Capital staff are almost totally devoted to building better products.

However, clearly Private Equity employs a methodology, a process, an approach, an algorithm, so there is definitely some aspect of science and engineering in all of this.

To recognize that reality, the jargon for this entire strategy, nowadays, is to call it "financial engineering."

Look that up in your web search sometime, and you'll see that it's in fact quite advanced, and includes lots of subtleties such as tax codes, interest rates, etc.

Like I said at the beginning, I'm a software engineer, not a financial engineer.

However, like any engineer, I find pretty much all engineering interesting, and so I have a certain morbid fascination with the ways of the financial engineers in the world of Private Equity.

But, I'm a software engineer, so I really would prefer to Build a Better Product, and create value that way.

Sadly, the world isn't always run To Please Bryan.

Anna Wiener really hit the nail on the head

I assume that, by now, you've read Anna Wiener's superb short story: Uncanny Valley.

It's been a long time, a very long time, since I've read something that is so accurate about these times, these emotions, these moments, as Wiener's story is.

This is not to confuse confidence with pride. I doubt myself daily. I feel lucky to have this job; I feel desperately out of place. My previous boss — breezy and helpful, earnest in the manner of a man who in his early twenties bequeathed $4 million to disrupt libraries — had encouraged me to apply for the role; I had joined his publishing start-up too early and needed something new. “This is the next big company,” he had said. “It’s a rocket ship.” He was right. I had been banking on him being right. Still, there are days when all I want is to disembark, eject myself into space, admit defeat.

I laugh, I cry, I recognize myself, or people I know all too well, in every paragraph, in every sentence, in almost every word.

Here's Wiener's description of an awkward blind date with an out-of-town developer. It so perfectly captures the wild mood swings from desire to agony; it so perfectly lampoons the desperate energy of our frantic, over-scheduled lives; and it so perfectly skewers the social awkwardness of our we-live-vicariously-through-our-social-media-selves, that my jaw drops with astonishment.

“There’s no menu, so you can’t just order, you know, a martini,” the developer says, as if I would ever. “You tell the bartender three adjectives, and he’ll customize a drink for you accordingly. It’s great. It’s creative! I’ve been thinking about my adjectives all day.”

What is it like to be fun? What is it like to feel like you’ve earned this? I try to game the system by asking for something smoky, salty, and angry, crossing my fingers for mezcal; it works. We lean against a wall and sip. The developer tells me about his loft apartment in the Mission, his specialty bikes, how excited he is to go on weeknight camping trips. We talk about cameras and books. We talk about cities we’ve never visited. I tell him about the personal-shopper service my coworkers all signed up for, how three guys came into work wearing the same sweater; he laughs but looks a little guilty.

I haven't read much of Wiener's work before, but I certainly shall be hoping to read more of it in the future.

Go Foxes!

They can clinch it tomorrow, though it will take a fine performance.

Meanwhile, America has finally taken notice: The Most Shocking Championship in the History of Professional Sports

The Premier League has no salary cap, no draft, and no help for the little guy. At the beginning of the 2015 season, you could get 30-to-1 odds on a Royals title. For Leicester City, those odds were 5,000 to 1. If it wins the title, it may be the most unlikely successful single event bookies have ever taken bets on. The English bookmakers William Hill will also take bets at 5,000 to 1 on Elvis being found alive, Kim Kardashian winning the presidency in 2020, and Barack Obama playing cricket for the English national team.

I've only seen a few of their games. It's entertaining, though a bit hard to watch.

Defenders, midfielders, and forwards stay close together and defend as a 10-man unit, forcing opponents to navigate three lines of players in quick succession if they want to get to goal. That compactness helps Leicester press and funnel the opposition wide

Who knows what will happen in the future, but one thing is sure: enjoy this great story for the time being.

This is the greatest underdog story in modern sports history. The barriers to entry were significantly lower when the English club Nottingham Forest won promotion, the title, and the European Cup in three consecutive seasons from 1976–79. Rulon Gardner and the 1980 U.S. Olympic hockey team both pulled off amazing upsets, but those were one-shot deals. They didn’t outlast their opponents in a best-of-38 series.

Go Foxes!

A Bear's Eye View of Yellowstone

This interactive multimedia feature on the National Geographic website is FABULOUS: A Bear's Eye View of Yellowstone.

Enjoy.

Or, if you like your animal videos a little bit cuter, try this: This Dog Passes a Soccer Ball Better Than You Do

But, really, the bear page is much, much better: way to go, NatGeo!

EuroSys 2016 links

Last week was the EuroSys 2016 conference, and once again the conference had an extremely interesting program of sessions.

A few that caught my particular eye...

• The Linux scheduler: a decade of wasted cores

  Cores may stay idle for seconds while ready threads are waiting in runqueues. In our experiments, these performance bugs caused many-fold performance degradation for synchronization-heavy scientific applications, 13% higher latency for kernel make, and a 14-23% decrease in TPC-H throughput for a widely used commercial database. The main contribution of this work is the discovery and analysis of these bugs and providing the fixes.

• Hold 'em or fold 'em?: aggregation queries under performance variations

  Due to large performance variations in clusters, some processes are slower. Therefore, aggregators are faced with the question of how long to wait for outputs from processes before combining and sending them upstream. Longer waits increase the response quality as it would include outputs from more processes. However, it also increases the risk of the aggregator failing to provide its result by the deadline.

• A high performance file system for non-volatile main memory

  In this paper, we propose HiNFS, a high performance file system for non-volatile main memory. Specifically, HiNFS uses an NVMM-aware Write Buffer policy to buffer the lazy-persistent file writes in DRAM and persists them to NVMM lazily to hide the long write latency of NVMM. However, HiNFS performs direct access to NVMM for eager-persistent file writes, and directly reads file data from both DRAM and NVMM as they have similar read performance, in order to eliminate the double-copy overheads from the critical path.

• pVM: persistent virtual memory for efficient capacity scaling and object storage

  pVM extends the OS virtual memory (VM) instead of building on the VFS and abstracts NVM as a NUMA node with support for NVM-based memory placement mechanisms. pVM inherits benefits from the cache and TLB-efficient VM subsystem and augments these further by distinguishing between persistent and nonpersistent capacity use of NVM. Additionally, pVM achieves fast persistent storage by further extending the VM subsystem with consistent and durable OS-level persistent metadata.

• zExpander: a key-value cache with both high performance and fewer misses

  In this paper, we show that, by leveraging highly skewed data access pattern common in real-world KV cache workloads, we can both reduce miss ratio through improved memory efficiency and maintain high performance for a KV cache. Specifically, we design and implement a KV cache system, named zExpander, which dynamically partitions the cache into two sub-caches. One serves frequently accessed data for high performance, and the other compacts data and metadata for high memory efficiency to reduce misses.

• A study of modern Linux API usage and compatibility: what to support when you're supporting

  This paper presents a study of Linux API usage across all applications and libraries in the Ubuntu Linux 15.04 distribution. We propose metrics for reasoning about the importance of various system APIs, including system calls, pseudo-files, and libc functions. Our metrics are designed for evaluating the relative maturity of a prototype system or compatibility layer, and this paper focuses on compatibility with Linux applications.

• POSIX abstractions in modern operating systems: the old, the new, and the missing

  Little has been done to measure how and to what extent traditional POSIX abstractions are being used in modern OSes, and whether new abstractions are taking form, dethroning traditional ones. We explore these questions through a study of POSIX usage in modern desktop and mobile OSes: Android, OS X, and Ubuntu. Our results show that new abstractions are taking form, replacing several prominent traditional abstractions in POSIX.

• Efficient queue management for cluster scheduling

  Job scheduling in Big Data clusters is crucial both for cluster operators' return on investment and for overall user experience. In this context, we observe several anomalies in how modern cluster schedulers manage queues, and argue that maintaining queues of tasks at worker nodes has significant benefits. On one hand, centralized approaches do not use worker-side queues. Given the inherent feedback delays that these systems incur, they achieve suboptimal cluster utilization, particularly for workloads dominated by short tasks. On the other hand, distributed schedulers typically do employ worker-side queuing, and achieve higher cluster utilization. However, they fail to place tasks at the best possible machine, since they lack cluster-wide information, leading to worse job completion time, especially for heterogeneous workloads.

You'll notice I've been particularly interested in Non Volatile Memory papers recently. It's a trendy thing, and I'm trying to get my head around it...

Another collection of links

Spring has sprung, and the sun is shining; but I'm inside reading?

Let me just finish reading this one more article, and then I'll go out...

• Ford hedges to deal with disruption

  If you have wondered how one company can have a separate car and mobility business you have already understood the problem. Cars are subservient to mobility, but Ford is treating them as distinct. Put simply, if disruption is coming from a new way of putting the parts of the system together, the solution cannot be to keep them separate. Yet that is what Ford, like so many before them, are doing.

• The engineer’s engineer: Computer industry luminaries salute Dave Cutler’s five-decade-long quest for quality

  Cutler, 74, who still comes to his office each day on Microsoft’s sprawling Redmond, Washington, campus, has shaped entire eras: from his work developing the VMS operating system for Digital Equipment Corporation in the late ‘70s, his central role in the development of Windows NT – the basis for all major versions of Windows since 1993 – to his more recent experiences in developing the Microsoft Azure cloud operating system and the hypervisor for Xbox One that allows the console to be more than just for gaming.

  It's interesting to see how Cutler's reputation has mellowed over the decades. When I was still a young sprout in the programming world, back in the early 1980's, Cutler had an absolutely ferocious reputation for his temper. The closest that the article gets to that, though, is the quote from Gordon Bell: "Dave is the ultimate competitor. He really wants to win." I do remember, however, that Cutler's reputation for excellence, however, was just as strong then as it is now.

• How Etsy Formats Currency

  Formatting currency for international members is hard. Etsy supports browsing in 9 languages, 23 currencies, and hundreds of regions. Luckily, we don’t have to figure out the right way to format in all of these combinations, because the nice folks at CLDR have done it for us. CLDR is a massive database of formatting styles that gets updated twice a year. The data gets packaged up into a portable library called libicu. Libicu is available everywhere, including mobile phones. If you want to format currency, you can use CLDR data to do it.

• Why aren’t we using SSH for everything?

  It means that SSH has authentication built into the protocol. When you join ssh-chat, not only do I know who you claim to be, but I can also permanently and securely attach an identity to your connection without any user intervention on your part. No signup forms, no clicking on links in your email, no fancy mobile apps.

• Five ways to paginate in Postgres, from the basic to the exotic

  PostgreSQL gives us a number of server-side pagination techniques that differ in speed, integrity (not missing records), and support for certain page access patterns. Not all methods work in all situations, some require special data or queries. Let’s consider the methods in order of generality, starting with those that work for any query, then those which require ordered data. We’ll conclude with some exotic methods which rely on PostgreSQL internals.

• Curing Our Slack Addiction

  your first instinct would be to switch to Slack and ask. And since everyone’s addiction was as strong as yours, you were sure to get someone’s attention.

  All of these interactions would happen in Slack, despite there being many other tools that are better suited. Tools like bug trackers and wikis would allow answers to be preserved so future questions wouldn’t even have to be asked but they weren’t as fun.

• Why the Unicorn Financing Market Just Became Dangerous ... For All Involved

  All Unicorn participants — founders, company employees, venture investors and their limited partners (LPs) — are seeing their fortunes put at risk from the very nature of the Unicorn phenomenon itself. The pressures of lofty paper valuations, massive burn rates (and the subsequent need for more cash), and unprecedented low levels of IPOs and M&A, have created a complex and unique circumstance that many Unicorn CEOs and investors are ill-prepared to navigate.

• Silicon Valley’s $585 Billion Problem

  All signs point to a continued slowdown in tech IPO activity in 2016, says Kathleen Smith, a principal at Renaissance and the company's manager of IPO-focused ETFs. She says it won't take long for the unicorns to feel the chill as well. "What's happening now is just going to take the bottom out of these private valuations, many of which are imaginary," says Smith. "And this valuation reset is going to have a very negative effect on new funding."

• Apple's Organizational Crossroads

  Apple employs what is known as a "unitary organizational form" — U-form for short — which is also known as a "functional organization." In broad strokes, a U-form organization is organized around expertise, not products: in the case of Apple, that means design is one group (under Ive), product marketing is another (under Schiller), and operations a third (under Williams, who is also Chief Operating Officer). Other areas of expertise represented by the members of Apple’s executive team include Software Engineering (Craig Federighi), Hardware Engineering (Dan Riccio), and Hardware Technologies (Johny Srouji).

  What is most striking about that list is what it does not include: the words iPhone, iPad, Mac, or Watch. Apple’s products instead cut across the organization

• Why Are America's Most Innovative Companies Still Stuck in 1950s Suburbia?

  When Apple finishes its new $5 billion headquarters in Cupertino, California, the technorati will ooh and ahh over its otherworldly architecture, patting themselves on the back for yet another example of "innovation." Countless employees, tech bloggers, and design fanatics are already lauding the "futuristic" building and its many "groundbreaking" features. But few are aware that Apple's monumental project is already outdated, mimicking a half-century of stagnant suburban corporate campuses that isolated themselves—by design—from the communities their products were supposed to impact.

• Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving

  Most of us took mathematics courses from mathematicians—Bad Idea!

  Mathematicians see mathematics as an area of study in its own right. The rest of us use mathematics as a precise language for expressing relationships among quantities in the real world, and as a tool for deriving quantitative conclusions from these relationships. For that purpose, mathematics courses,as they are taught today, are seldom helpful and are often downright destructive.

• Anthropic Capitalism And The New Gimmick Economy

  If you look at your news feed, you will notice that the economic news no already longer makes much sense in traditional terms. We have strong growth without wage increases. Using Orwellian terms like "Quantitative Easing" or "Troubled Asset Relief", central banks print money and transfer wealth to avoid the market's verdict. Advertising and privacy transfer (rather than user fees) have become the business model of last resort for the Internet corporate giants. Highly trained doctors squeezed between expert systems and no-frills providers are moving from secure professionals towards service sector-workers.

  Capitalism and Communism which briefly resembled victor and vanquished, increasingly look more like Thelma and Louise; a tragic couple sent over the edge by forces beyond their control. What comes next is anyone's guess and the world hangs in the balance.

• Backing up 18 years in 8 hours

  When I was a teenager, whenever a hard disk needed replacement, I'd pull the old drive and shove it in my closet. There they sat, some for over a decade, until I turned them back on last month.

• Farewell, App Academy. Hello, Airbnb. (Part I) (and Farewell, App Academy. Hello, Airbnb. (Part II) )

  As an instructor at App Academy I ceaselessly pushed my students to negotiate hard, without fear of being rejected, looking stupid, or being perceived as greedy. Employers negotiate even harder and with more power behind them, and so it’s up to candidates to tip the scales back in the direction of employees. As absurd as it seemed now, given an offer of 220K, I had to take my own advice and ask for more.

Another game industry controversy

Well, it looks like the Internet is about to burst into fire again.

The video-gaming industry is immense and is an extremely important part of the overall tech industry, but they have also been a problem-plagued part of the industry. The well-publicized "GamerGate" is only the most well-known of a number of serious problems dating back for decades.

Anyway, this week comes this spark in a very dry forest: Game developers must avoid the ‘wage-slave’ attitude

Many modern game developers have embraced a culture of victimology and a bad attitude toward their chosen vocations. They complain that the long hours and personal sacrifices great games require are a consequence of poor management. They want to pretend that they can turn an inherently entrepreneurial endeavor like game development into a 9-to-5 job. Somehow, these people have managed to adopt a wage-slave attitude toward one of the most remarkable and privileged careers in the world.

Who knows why VentureBeat thought this screed was worth hosting on their site, but up it went last weekend.

It wasn't long before it was noticed, and people started to react.

One of the first to react was Jason Schreier at Kotaku: Game Industry Veteran Writes Horrifying Article In Defence Of Poor Working Conditions

The VentureBeat article is tough to read. But as remarkably bad an argument as it makes, it’s also an insightful peek into the ethos that’s led to systemic problems like frequent crunch and unfair pay. Twisted arguments like “This is art, not work” and “You should just feel lucky to be here” have been used for decades to deny game developers of their right not just to living wages, but to have lives outside of their workplaces.

The discussion in the comments is fascinating for a glimpse into the emotions and passion behind the issue.

For a slightly-more-distanced view, here's Vox with one of their great explainers: Everything wrong with Silicon Valley culture in one gross presentation, including a thorough and insightful deconstruction of the language of St. John's writings:

In many ways, St. John's attitude toward recruiting is typical: He advises employers to present both their current and potential engineers with tangible goals, meaningful challenges, and lucrative rewards. What's not typical is the way he advises selecting those engineers, especially with regard to the skills and character traits he believes employers should prioritize.

For starters, with few exceptions St. John believes they should focus almost exclusively on men. We know this because he emphasizes recruiting and retaining engineers' "wives and girlfriends," because they are the key to whether an engineer stays at a company or quits. Oy.

So much, so video-gaming-industry.

But then things took a very interesting turn late last week when St. John's own daughter took to the airwaves to comment: I am Alex St. John’s Daughter, and He is Wrong About Women in Tech

But as a woman, to enter this privileged position in the first place I had to face a lot of difficult situations(And no, none of those situations involved a wrestle with my “victim complex”). The experience has left me with more than a few opinions about my father’s views on this subject (which are exceptionally vile and wrong).

Her entire essay is fascinating, but I was particularly struck by her analysis of the complex social implications of certain organizational practices that appear to be gaining some ground, such as Alex St. John's recommendation to route female employees into positions in technical writing, QA, and technical support:

Widely held beliefs like these are playing a huge role in hindering women from continuing as engineers. While many of these “more social” roles may be high paying, they remove truly technical women from technical jobs, furthering the imbalance. This directly impacts women later in their careers as it has been shown that technical positions are more likely to lead to senior roles in the industry. My Father’s suggestion to continue the practice of “promoting” women out of engineering roles will only further reinforce gender norms in the workplace and ultimately harm the supply of senior female technical executives.

Sadly, it's impossible to overlook how St. John reacted to this, starting first with his defensive follow-up: Enslaving the Masses, in which he doubles down:

Today it seems that instead of training the defeatist and unprofessional attitudes out of young technology professionals they are being deliberately conditioned IN to them. I know that a lot of those new generation technologists in the valley think that their brand of permissive lassitude is some form of progress, but it’s not, it’s just a lot of wealth and success in the Valley enabling a generation of people to grow up thinking that success is easy, that they are entitled to it and that there will be no consequences for embracing bad attitudes towards work. It’s a sad condition, I actually don’t think it’s curable once they’ve reached the valley and found a community of people who actively reinforce the behaviors but I’m still determined to at least try to make sure that it’s called out and characterized as clearly as possible for the tiny few who maybe eventually mange to save themselves from it.

If that wasn't bad enough, he continued yesterday, with perhaps the most bizarre "apology" you'll read in a long time: I Apologize

I shrugged, Sharon already worked late anyway, why not? A few days later our HR manager informed me that all of the goth QA team also only wanted to work late at night because they ALSO suffered from sensitivity to day light. I pointed out that we kind of needed them around during some portion of the regular work-day to communicate with the engineering teams and product teams. We tried to figure out if there was any part of the regular work day when it was dark out but of course there was none.

Oh, dear.

The tech industry is changing, in some ways, but the change is coming slowly, and unevenly.

There are enormous, decades-old, very hard-to-solve problems here, but scorn, ridicule, and disdain are not the answer.

None of this stuff is easy to read, and I don't have any simple answers of my own.

But I think it starts with paying attention, thinking about it, and being willing to talk about it.

401K Fees

I don't often comment on other people's blogs, but this issue got me worked up enough that I "contributed to the discussion," FWIW: Profits in Finance.

This was the part I reacted to:

Expense ratios on actively managed mutual funds have remained stubbornly high. Even though more people switch into index funds every year, their overall market share is still low—about $2 trillion out of a total of $18 trillion in U.S. mutual funds and ETFs. Actively managed stock mutual funds still have a weighted-average expense ratio of 86 basis points.

Why do people pay 86 basis points for a product that is likely to trail the market, when they could pay 5 basis points for one that will track the market (with a $10,000 minimum investment)? It’s probably because they think the more expensive fund is better.

And here is the comment I left, in case it disappears from the other site:

I think both you and Noah are missing the reason, a bit. Most people that I know who have investments in high-fee funds, actually have no choice. The people I know in this situation are participating in these funds through their 401K plan, and their 401K plan offers only high-fee funds. Myself, I’ve worked at more than half-a-dozen companies over the last 25 years, both large companies and small companies, and in only ONE of those situations was there even a single low-fee fund available.

With 401K plans, employees have no choice to go outside the funds offered by their plan. If their plan offers only high-fee funds, their only alternative is to forego the 401K plan entirely, which is a far worse choice then enduring the high fee.

Surprisingly to me, I don't see this issue getting much attention.

I'm not sure why that is.

But I suppose we each have our own hobby-horse to ride...

Read. Learn. Repeat.

At least, try to.

• Gone In Six Characters: Short URLs Considered Harmful for Cloud Services

  URL shorteners such as bit.ly and goo.gl perform a straightforward task: they turn long URLs into short ones, consisting of a domain name followed by a 5-, 6-, or 7-character token. This simple convenience feature turns out to have an unintended consequence. The tokens are so short that the entire set of URLs can be scanned by brute force. The actual, long URLs are thus effectively public and can be discovered by anyone with a little patience and a few machines at her disposal.

  Today, we are releasing our study, 18 months in the making, of what URL shortening means for the security and privacy of cloud services. We did not perform a comprehensive scan of all short URLs (as our analysis shows, such a scan would have been within the capabilities of a more powerful adversary), but we sampled enough to discover interesting information and draw important conclusions.

• Introducing Facebook's new terrestrial connectivity systems — Terragraph and Project ARIES

  Facebook's Connectivity Lab is working on a range of new technology solutions to help connect the unconnected and improve the experience of the underserved. Today we announced two new terrestrial systems focused on improving the speed, efficiency, and quality of internet connectivity around the world — Terragraph and Project ARIES (Antenna Radio Integration for Efficiency in Spectrum).

• Security Protocols 2016

  I’m at the 24th security protocols workshop in Brno (no, not Borneo, as a friend misheard it, but in the Czech republic; a two-hour flight rather than a twenty-hour one).

• The Twelve-Factor App

  This document synthesizes all of our experience and observations on a wide variety of software-as-a-service apps in the wild. It is a triangulation on ideal practices for app development, paying particular attention to the dynamics of the organic growth of an app over time, the dynamics of collaboration between developers working on the app’s codebase, and avoiding the cost of software erosion.

  Our motivation is to raise awareness of some systemic problems we’ve seen in modern application development, to provide a shared vocabulary for discussing those problems, and to offer a set of broad conceptual solutions to those problems with accompanying terminology. The format is inspired by Martin Fowler’s books Patterns of Enterprise Application Architecture and Refactoring.

• Introducing Ignition: The new CoreOS machine provisioning utility

  Ignition is a new machine provisioning utility designed to solve the same problems as coreos-cloudinit while adding a host of new capabilities with clearer semantics. At the the most basic level, Ignition is a tool for manipulating disks during early boot. This includes partitioning disks, formatting partitions, writing files, and configuring users.

• Here's The Programming Game You Never Asked For

  I'm thinking about this because I believe there's a strong connection between programming games and being a talented software engineer. It's that essential sense of play, the idea that you're experimenting with this stuff because you enjoy it, and you bend it to your will out of the sheer joy of creation more than anything else.

• Earthquake mitigation at Oakland City Center

  During a moderate-to-major event, say magnitude 5-1/2 or larger, the Clorox building will shimmy back and forth, and so will the buried BART station. Because of their different sizes and dimensions, they won’t move in unison. Without the gasket in the pavement, the tiles in the rigid pavement will buckle and shatter and fly in all directions, leaving one more mess to clean up that will probably fester for years.

  The gasket promises to prevent that. If you’re here when the next sizeable earthquake hits, and you have the presence of mind (not guaranteed!), watch it work. The free side of the steel ribbon should slide over the ground while the gasket cushions the two sides of the cut beneath it.

• Collapse of kelp forest imperils North Coast ocean ecosystem

  Scientists blame the current situation on what some have called “a perfect storm” of large-scale environmental impacts dating back to 2011, when a harmful algal bloom off the Sonoma Coast released toxins that killed large numbers of red abalone and took a toll on other invertebrates.

  In 2013, an eruption of sea star wasting disease along the West Coast eliminated vast quantities of starfish, affecting about 20 species to varying degrees, including at least two with pivotal roles in the food web.

  Starfish are the primary predators of purple urchins, particularly in areas like Northern California, where sea otters — another significant urchin eater — have not rebounded as they have begun to do elsewhere on the coast.

  Without such predators to keep them in check, the density of purple urchins on the North Coast reportedly now is more than 60 times that observed in the past, Catton said.

Things to read when you're reading

That is, when you're not following the Warriors, or taking a walk on a beautiful spring day...

• Google Compute Engine Incident #16007

  on this occasion our network configuration management software detected an inconsistency in the newly supplied configuration. The inconsistency was triggered by a timing quirk in the IP block removal - the IP block had been removed from one configuration file, but this change had not yet propagated to a second configuration file also used in network configuration management. In attempting to resolve this inconsistency the network management software is designed to ‘fail safe’ and revert to its current configuration rather than proceeding with the new configuration. However, in this instance a previously-unseen software bug was triggered, and instead of retaining the previous known good configuration, the management software instead removed all GCE IP blocks from the new configuration and began to push this new, incomplete configuration to the network.

  One of our core principles at Google is ‘defense in depth’, and Google’s networking systems have a number of safeguards to prevent them from propagating incorrect or invalid configurations in the event of an upstream failure or bug. These safeguards include a canary step where the configuration is deployed at a single site and that site is verified to still be working correctly, and a progressive rollout which makes changes to only a fraction of sites at a time, so that a novel failure can be caught at an early stage before it becomes widespread. In this event, the canary step correctly identified that the new configuration was unsafe. Crucially however, a second software bug in the management software did not propagate the canary step’s conclusion back to the push process, and thus the push system concluded that the new configuration was valid and began its progressive rollout.

• Evaluating Database Compression Methods: Update

  This blog post is an update to our last post discussing database compression methods, and how they stack up against each other.

  When Vadim and I wrote about Evaluating Database Compression Methods last month, we claimed that evaluating database compression algorithms was easy these days because there are ready-to-use benchmark suites such as lzbench.

  As easy as it was to do an evaluation with this tool, it turned out it was also easy to make a mistake. Due to a bug in the benchmark we got incorrect results for the LZ4 compression algorithm, and as such made some incorrect claims and observations in the original article.

• DIY Jepsen Testing CockroachDB

  When you work on Jepsen, you usually only look at problems (like claims to serializability and linearizability) that involve a combination of A, C, and I in a distributed environment because you can assume the database engine already worked very hard on A, C, I, and D individually. After all, these engines have been around for a while and have plenty of users already who would have complained earlier if their transactions were not atomic or durable, or if they could see gross consistency anomalies.

  In contrast, CockroachDB is still rather young, so a few more ACID checks were warranted. We figured we could learn how to use Jepsen by implementing a few simple tests first.

• How to write a Bloom filter in C++

  Bloom filters are data structures which can efficiently determine whether an element is possibly a member of a set or definitely not a member of a set. This article will cover a simple implementation of a C++ bloom filter. It’s not going to cover what bloom filters are or much of the math behind them, as there are other great resources covering those topics.

• Frog and Toad are Cofounders

  The characters you loved as a child, all grown up and working in San Francisco

• Visual Studio Code 1.0!

  We wanted to build a native development tool that developers could install and use anywhere, for any source code. And, from our experience, we believed that it was important to not just have an editor, but one that could help developers accomplish their most common tasks: navigating code, debugging, and working with Git. And, so, Visual Studio Code was born.

  Being built on web technologies made it easy to host the tool in a native cross-platform shell. We decided early on to use, and contribute to, a number of open source technologies – including GitHub’s great Electron shell, which combines web and native UI with a Node.js API.

• When Bitcoin Grows Up

  In time, even the US joined the system of state-backed money dispensed through a central bank. This is the system we still have everywhere in the developed world today. The reason a lot of people are excited about bitcoin and its associated technologies is that for the first time there is a genuine possibility of real change in this area. Money has evolved in jumps, from the invention of writing to the invention of the balance sheet and the bank to the creation of the central bank, with all of these changes being variations on the theme of money as a register of credits and debits. And we’re now at a point when another jump is possible.

• Statistics for Software

  Questions about software performance cannot be answered, or even appropriately articulated, without statistics.

  Yet most developers can’t tell you much about statistics. Much like math, statistics simply don’t come up for typical projects. Between coding the new and maintaining the old, who has the time?

• Remote Procedure Call

  This is one post in a series about programming models and languages for distributed computing that I’m writing as part of my history of distributed programming techniques.

  Remote Procedure Call (RPC) is a general term for executing a subroutine in a different address space without writing the actual code used to perform the remote execution. To provide an example, we can imagine a user wishing to invoke the random number generator function on another machine, but, the only difference between the local and remote invocation is supplying an additional node identifier where it should occur.

• Questions to Ask a Potential Tech Employer

  This is a list of things to ask a potential employer when you're interviewing for a technology job. It's compiled from books, the web, contributors, and personal experience. Asking good questions will not only give you a better idea of what a company is like, but they will respect you for digging (if they don't -- run!).

• Ominous new cracks found on Bay Bridge rods

  Tiny cracks found on some of the rods on the new Bay Bridge tower potentially endanger the rest of the more than 400 remaining fasteners that secure the tower to the foundation in an earthquake, Caltrans officials said Tuesday.

  They also acknowledged that one of four high-strength tower anchor rods they have examined apparently snapped after it was exposed to water and became brittle. That was the same headache that cost the agency $45 million to fix in 2013 when 32 rods on seismic stabilizers failed after sitting in water.

  Brian Maroney, Caltrans’ chief engineer on the bridge project, said the microscopic cracks found on at least two rods lead him to believe that the problem could be widespread and that the rods could be repaired or even replaced if necessary. Nearly all the rods have been exposed to water, and about a quarter of them have sleeves that routinely flood with bay water within days of being drained.

• The secret rules of the internet: The murky history of moderation, and how it’s shaping the future of free speech.

  Mora-Blanco is one of more than a dozen current and former employees and contractors of major internet platforms from YouTube to Facebook who spoke to us candidly about the dawn of content moderation. Many of these individuals are going public with their experiences for the first time. Their stories reveal how the boundaries of free speech were drawn during a period of explosive growth for a high-stakes public domain, one that did not exist for most of human history. As law professor Jeffrey Rosen first said many years ago of Facebook, these platforms have "more power in determining who can speak and who can be heard around the globe than any Supreme Court justice, any king or any president."

Things I found while cleaning out my sock drawer

It's been quiet recently. Everybody is doing their taxes, I guess.

• Saving 13 Million Computational Minutes per Day with Flame Graphs

  For those unfamiliar with flame graphs, the default visualization places the initial frame in a call stack at the bottom of the graph and stacks subsequent frames on top of each other with the deepest frame at the top. Stacks are ordered alphabetically to maximize the merging of common adjacent frames that call the same method. As a quick side note, the magenta frames in the flame graphs are those frames that match a search phrase. Their particular significance in these specific visualizations will be discussed later on.

• How a Hypothesis Can Be Neither True Nor False

  Kurt Gödel described a model that satisfies the axioms of set theory, which does not allow for an infinite set to exist whose size is between the natural numbers and the real numbers. This prevented the Continuum Hypothesis from being disproven. Remarkably, some years later, Paul Cohen succeeded in finding another model of set theory that also satisfies set theory axioms, that doesallow for such a set to exist. This prevented the Continuum Hypothesis from being proven.

  Put another way: for there to be a proof of the Continuum Hypothesis, it would have to be true in all models of set theory, which it isn’t. Similarly, for the Hypothesis to be disproven, it would have to remain invalid in all models of set theory, which it also isn’t.

• Site Reliability Engineering: How Google Runs Production Systems

  In this collection of essays and articles, key members of Google’s Site Reliability Team explain how and why their commitment to the entire lifecycle has enabled the company to successfully build, deploy, monitor, and maintain some of the largest software systems in the world. You’ll learn the principles and practices that enable Google engineers to make systems more scalable, reliable, and efficient—lessons directly applicable to your organization.

• Robots against robots: How a Machine Learning IDS detected a novel Linux Botnet

  Still unknown and hidden.

  Could not be detected by usual protections.

• Lawful Hacking and Continuing Vulnerabilities

  Whatever method the FBI used to get into the San Bernardino shooter's iPhone is one such vulnerability. The FBI did the right thing by using an existing vulnerability rather than forcing Apple to create a new one, but it should be disclosed to Apple and patched immediately.

• Attack of the Week: Apple iMessage

  it's worth noting that the security of a text messaging protocol may not seem like the most important problem in computer security. And under normal circumstances I might agree with you. But today the circumstances are anything but normal: encryption systems like iMessage are at the center of a critical national debate over the role of technology companies in assisting law enforcement.

• Down from Eight Miles High… and Landing!

  Neurensic has some lucky timing and some serious financial market Quant talent. They put together a Machine Learning-based product to find financial market cheaters – and just in time to match the new Dodd-Frank compliance rules. Their timing is impeccable; suddenly everybody who is responsible for traders’ good behavior (i.e. the Banks) wants their stuff. Their first generation product is just coming out, and I plan on bring my own skill and H2O’s Big Data and ML abilities to dramatically up Neurensic’s game. The Goal: keeping the Stock Market (well, all financial markets) safe for everybody.

• The Sun Rises--But the Sun Also Sets...

  Ever since I was six years old or so, my overriding goal has been to become as smart as possible--not to become the smartest person around (as it was all was clear that that was simply not possible), but as smart as I could be. That seemed to be a very fun way to live one's life. And it is.

• The Rain Barrel Is Only the Beginning of the West’s Water Wars

  For years now, Coloradans have been fighting over the seemingly innocuous rain barrel. Environmentally conscious and thrifty residents say they should have the right to catch as much rain as they please, while agricultural interests argue that doing so is tantamount to stealing water from its rightful owners. The same battle has played out throughout the West, but Colorado is the last remaining state where barrels are banned outright—for a few more days, anyway. On April 1, the state legislature passed a bill to permit residents to catch rain in measured quantities, and Governor John Hickenlooper is expected to sign it.

  Coloradans will no longer have to hide their downspouts and barrels for fear of being fined, but the issue is hardly settled. The forthcoming law keeps historical water law intact, while opening a crack in the once unbreakable water doctrine by allowing two 55-gallon barrels per residential property.

• Number Of Wild Tigers Increases For First Time In 100 Years

  The WWF cites factors such as better survey processes and enhanced protections in explaining the gains. But it adds that the world's tigers remain threatened by shrinking habitats in Asia and that they are also a prime target for poachers.

  "Every part of the tiger — from whisker to tail — is traded in illegal wildlife markets, feeding a multi-billion dollar criminal network," the organization says.

The "Classic Programmer Paintings" are wonderful

I'm sure you've already seen them, but just in case: Classic Programmer Paintings.

Concern for those around you

My grand-daughter was worried that the fairies who inhabit my house weren't finding enough crumbs, now that the kids have moved out and don't visit like they used to.

So she made them a pepperoni pizza and an ice cream sundae.

Possibly the fairies were busy last night.

I'll leave it out again tonight for them to find.