Big data, Collaboration, Economy, retail

The Retail “Apocalypse”

Retail is changing.  

US online sales has been growing 15.6% year over year at roughly 4x the rate of overall retail market, amounting to 15% of all retail sales in 2019.  Depending on the time of year, location and category, the change is even more pronounced.  

1010data’s report shows that while overall Black Friday sales held to similar levels year over year, representing almost 30 percent of the week’s sales, share of in-store channel has dropped from an average of 83 percent of spend in November, 2014 to an average of 68 percent in November 2019.  

For books about 15% digital penetration led to consolidation and eventually bankruptcy of physical bookstore chains. Today for apparel and electronics, share of online sales has already surpassed 30%. On average, share of online grocery sales is still in single digits, but it varies greatly by locale, from a high of 12% in New York City to 5% in San Francisco and 2% in Des Moines. 

While improved delivery times and ease of returns made a significant change in consumer attitude towards online shopping overall, for groceries growth has been slow due to thin margins with an average item price of around $3 and 30% gross margin, leaving only $0.90 to for all of handling, selling and delivery. With KPMG predicting self-driving delivery vehicles to reduce the cost of delivery to between 4 and 7 cents per mile, financially viable online grocery businesses will be well within reach in the next few years.  

And I’m not talking about distant future here. A few months ago, UPS became the first company to receive FAA certification to build a “drone airline”. Certification allows them to fly drones out of operator line of sight, during day or night, over people and with cargo weighing more than 55 pounds. Soon UPS trucks will become hubs for swarms of drones, while in densely populated areas, drones will deliver directly from distribution centers to homes.  While waiting for its own drone services business’ FAA approval of course Amazon wasn’t resting on its laurels, piloting package delivery with their Scout autonomous vehicles. Startups like Cleveron, Postmates and Starship are also building their own commercial robots for last-mile delivery. This all spells trouble for companies like Instacart and retailers that plan on relying on them to solve their delivery problem in the longer run. 

Another and often complimentary approach to reducing last-mile distribution costs is micro-fulfillment centers for deliveries and pickup orders which make 1-day, even 1-hour delivery feasible. California-based startup Farmstead has made over 100,000 deliveries in San Francisco Bay Area in the past 3 years and recently announced expanding its footprint into Carolinas. Their micro-fulfillment centers cost 1% of an average supermarket to build, have much smaller footprints, allowing for much larger number of locations, closer to customers.  UK’s Ocado having shown that it is possible turn profit as an online-only supermarket with its $2+ Billion annual revenue is starting to transform itself to a technology company by licensing its automated fulfillment center tech. Truepill, Alto Pharmacy, Nimble RX and Capsule are applying the same formula to pharmacy, delivering not only your regular prescriptions but also offer same-day delivery at no extra cost so you don’t have to endure long pharmacy lines while you’re sick. 

Many large US retailers are already in various stages of pilot implementations. Kroger through a joint venture with Ocado, is building 20 fulfillment centers for online orders and have so far committed to facilities near Dallas, Cincinnati, Orlando and Atlanta.  Meijer recently announced that it will begin testing micro-fulfillment with logistics company; Dematic. Albertsons is piloting a fully automated micro-fulfillment center with the robotics company Takeoff TechnologiesFabric claims to have built the world’s smallest fulfillment center, which can process up to 600 orders per day out of 6,000 square feet. That is roughly twice the number of orders per square foot an average brick-mortar store would generate per Readex Research’s 2019 Annual Retailer Survey.  Considering average online grocery basket size is also roughly twice1 that of brick-and-mortar, assuming a fulfillment center operating at full capacity, this translates to roughly 4x revenue per square foot compared to traditional brick-and-mortar store setup.  When you factor in the cost savings of distribution facilities relative to commercial retail space, the online model becomes even more compelling. 

Finally, many staples we’re used to buying in stores are very well suited to planned, automatic replenishment. You can schedule deliveries for your cereal and peanut butter once a week, toilet paper once a month and adjust as needed to whatever cadence best suits your household. If you own a loyalty card and use the same retailers consistently, it is only a matter of time before this gets fully automated. Thanks to machine learning, it won’t be too long before Alexa reminds you if you’d like your monthly pantry items delivered to your home or pick them up from the nearest Amazon Locker on your drive back from work. Harry’s, Dollar Shave ClubBarkbox are a few companies who have been successful with this model. Not to mention Farmstead, with most of its customers enrolled in a weekly subscription program to save money on staples like milk, eggs, bacon and most packaged goods. Predictability allows Farmstead to better optimize their supply chain, reduce waste and pass on those savings to their customers. Meal kit vendors have also benefited from such predictability resulting in reduced carbon footprint

What does all this mean for traditional brick-and-mortar retailers?  

UK-based Argos is a rare example from the catalog retail era that started in early 1900s. While almost all those retailers went bankrupt within the past 2 decades, Argos survived by transforming itself, with e-commerce click-and-collect and delivery options, now majority of its revenue being generated through online sales. Retailers of today have a similar choice but it is easier said than done. 

Transforming to an omni-channel retailer requires significant innovation and organizational change; a system-wide digital transformation including the supply chain: 

1. Articulate your vision. Executing across multiple channels harmoniously requires a highly concerted effort that might be difficult to adapt in organizations that are used to operating in silos. A strong vision from the point of view of the customer and their changing needs and expectations, not by new technology or existing organizational boundaries is key to aligning various stakeholders. 

2. Define your strategy.  Retailers need to consider their market and make their own decisions about business and operations as there is no single recipe for success. Different customer segments will value parts of the shopping experience differently, different products will align better with different distribution channels but there exists plenty of success stories to get inspiration from. 

  • Listen to your customers Internet gave customers a voice and they expect to be heard. This could be through customer support channels, social media, blogs, forums and indirect feedback through instrumenting of customer experiences. Brands that pay attention to customer feedback have more engaged customers, higher customer satisfaction scores and are able to identify new product opportunities.   Two great recent examples of this is Coca Cola and Soylent. Soylent made a name for itself with its “open-source” meal replacement products. They enabled their customer community to come up with DIY recipes and share with each other, some of which inspired recipes currently sold by the company. Coca Cola introduced the new Orange-Vanilla flavor as it was one of the most popular pairings based on the data from their Freestyle fountain dispensers. Accenture found 91% of consumers prefer to buy from brands that remember their choices and provide relevant offers and recommendations while 83% are willing to share their data to enable personalized experiences. While today customization typically means coupons in stores or product recommendations on e-commerce sites, as Coca Cola and StitchFix have shown, there are many more ways to personalize. 
  • Give customers a reason to come to your store For most customers, grocery shopping is a chore they would like to avoid if they could. Successful retailers find ways to draw them into their stores.  TJ Maxx and Lidl understand that people love the thrill of a treasure hunt.  Lidl, in addition to the usual meat, fruits and vegetables, offers a rotation of specials; “Lidl surprises” that are released every Monday and Thursday. As soon as they’re gone, they’re gone. Replace groceries with apparel and you end up with TJ Maxx’s formula. New selections at least once a month with deep discounts only available in store. Bonobos and Glossier took brick-and-mortar and turned it on its head in their successful showroom concepts, where customers visit the stores not to pick items off the shelves but for the experience, to try on products and get personalized fashion advice.  This is an approach that can be generalized to fast moving goods as well. Imagine yourself enjoying a wine flight, sampling food or even taking a cooking class in the store while robots in the automated warehouse at back of the store get your order ready for pick up on your way out. 
  • Meet your customers where they are Today’s customer has high expectations with regards to convenience and flexibility. They could be ordering through your website but exchanging at a local store, comparing product specs online but buying it in store or ordering online for in-store pickup. To be successful in the world of omni-channel retailing, a seamlessly integrated customer experience across all channels from physical stores, computers and mobile devices through apps, e-commerce sites and social media is required to deliver vastly improved customer experiences. It is important to understand what channels are most important for your customer base and start with those. Sometimes these might be usual suspects like subscriptions, free returns, curbside-pickup etc. but sometimes it might require thinking outside the box. For example, way ahead of its time in 2011 Tesco opened the world’s first virtual store in Seoul subway to help time-pressed commuters to shop on the go using their smartphones with same-day delivery. 
  • Improve your supply chain Grocery giant ALDI (owner of ALDI stores and Trader Joe’s) is known for its bargain prices. They primarily owe this to ~90% of their products being private label which means lower unit cost and reduced selection which in turn also means smaller store footprints. The British online grocery retailer Ocado operates no stores and does all home deliveries from its warehouses with an industry leading 0.02% waste. For Ocado fully automated warehouses not only allows to be price-competitive, it means a better customer experience, reduced waste and carbon-footprint.  Walmart spent $4 billion in 1991 to create Retail Link to better collaborate with its suppliers. Today there are many off-the-shelf platforms retailers can use for this purpose for a fraction of the cost. For customers, this means fewer out of stocks and the retailers an additional revenue stream through data monetization.  Retailers will need to redesign their supply chains based on services they want to offer which will often mean looking at the supply chain as many possible starting and ending points rather than a single flow to get products on the shelf. This also means a focus shift from on-shelf availability to dynamic trade-offs between availability, margins and delivery times, reallocating products across channels based on sell-through rates and even testing demand for a product in online before moving it to store shelves. This is only possible with a shared inventory and end-to-end visibility across all distribution channels.   

3. Assess infrastructure needs. Organizations must determine the necessary technology capabilities from data management and machine learning to in-store sensors, warehouse management and delivery capabilities to support their vision.  Being able to effectively execute on an omni-channel strategy requires a fully unified stack. Successful retailers use machine learning to watch consumer trends and customer feedback, personalize offers, manage product assortment, decide optimal distribution center locations, demand/inventory forecasting, understand user journey and marketing channel effectiveness, in-store IoT devices to react to user actions and inventory updates in real time, robotic automation to increase warehouse efficiency and sharing data with suppliers to more effectively manage inventory and enable timely direct-to-store shipments. Making all the components work together, integrating between different hardware and software components are often multi-year projects. 

4. Identify necessary organizational changesRetailers will need to restructure business processes and metrics, define rules for shipping products and allocating revenue between channels. If a gift is ordered from a website but exchanged at a local store where should the revenue go? What if the customer went to a store, saw a display model but product was out of stock then ordered from her smartphone to ship it to her home? With omni-channel retail blurring the lines, the right incentives need to be put in place for business success with all parties focusing on delivering customer value.  

Retail is changing but in a way, everything old is new again.

E-commerce sites are the new catalog stores, Alexa is not the name of the server who knows how you like your steak but a voice assistant who will know about almost all your buyer preferences, convenience stores will become vending machines you can walk into, and the milkman will be a robot.

All in all, it will be better for the consumer and less taxing on the environment.  

So why call it an apocalypse? It is the retail renaissance. 

Economy, Visualization

MACD with Tableau 8

To celebrate Tableau’s IPO I thought it would be appropriate to share a relevant visualization. Moving Average Convergence Divergence is a commonly used technical analysis indicator. At the heart of the analysis are two — a fast (short-period) and a slow (long-period) — exponential moving averages.  It is often accompanied by a candlestick chart.The main point of interest is the red and blue lines crossing each other.

MACD with Tableau 8

I used Dow Jones Industrial Average data for this visualization. EMAs are implemented as table calculations. It is posted on Tableau public so it is interactive and available to download. You can access it by clicking on the screenshot above.

One thing to keep in mind, when you’re calculating a 10 day EMA, you start with a 10 day simple moving average (SMA). The SMA for 10th day would be the first input as the data from the day before in your EMA calculation. This means for the days preceding, EMA would be NULL. I didn’t want to have nulls in my visualization so I calculated the first value outside Tableau and hardcoded it into the calculation and calculated the EMA for the rest of data points as usual.

Economy, Visualization

Is part-time work really the reason for the declining unemployment rate?

Today Bureau of Labor Statistics (BLS) announced unemployment numbers for September. Drop in unemployment rate stirred discussions as it makes Obama’s hand stronger in the presidential race. Some went as far as calling the numbers massaged[1] but majority of the skeptics claimed that the decline was due to increase in part-time jobs[2] and as part-time jobs get cut first when economic conditions worsen “It’s an employment recovery built on thin ice[3]”.

Let’s look at some of the claims and the data

According to Bloomberg Business Week “Some 582,000 more Americans, the most since February 2009, were working part-time last month because of slack business conditions or because those jobs were the only work they could find, according to Labor Department estimates.”

Labor Department estimates tell a different story. According to the data, 657,000 fewer Americans were working part-time for what BLS calls “economic reasons” in September 2012 compared to September 2011 and it was certainly not the most since February 2009.

According to Boston Globe, upon release of BLS data, John E. Silvia, chief economist for Wells Fargo & Co. said “September’s rise in part-time employment was due to college students returning to school and taking part-time jobs.” and it was the gain in part-time work that pushed U.S. jobless rate down.

According to the data, part-time jobs (secondary axis) are in decline in the past few months following a sharp increase between February and April. There’s a clear increase in full-time jobs (primary axis) in the recent months. Looking at this graph, it is hard to attribute the unemployment rate drop between August and September to part-time jobs.


So at least based on BLS household survey data, unemployment rate drop in the last two months is clearly because of the increasing number of full-time jobs. However to understand unemployment, it is important to look at how the rates are calculated.

The Labor Department uses two surveys, one to estimate payroll employment, and the other of households, to estimate unemployment. They both have their shortcomings and don’t always correlate with each other.

Household Survey, involving just 60,000 households, is subject to a relatively large sampling error.  Also in this survey, if a person has worked just one hour, they are counted as employed. Even people who did unpaid work in a family-owned enterprise (as long as it is more than 15 hours) are classified as employed. The payroll survey sample does not include new firms immediately. They are incorporated with a lag.

Another issue is the definition of labor force. Labor force measures are based on the civilian non-institutional population 16 years old and over and is made up of the employed and the unemployed. Persons are classified as “not in the labor force” if they do not have a job, and have not actively looked for work in the prior 4 weeks of the survey. So it is very easy to be unemployed but not counted as such.

Since U3 is how “official” unemployment rate has been calculated for years, people often refer to the changes of this parameter over time when they talk about unemployment. It makes sense for an apples-to-apples comparison, but there are other approaches considered to be closer to real unemployment such as U6. U6 considers involuntary part-time workers (those who took part-time jobs because they couldn’t find full time jobs) and marginally attached/discouraged workers (unemployed people who haven’t searched for jobs in more than 4 weeks) as part of unemployed labor force.


With either approach, unemployment appears to be in decline while with U6 we observe a considerably higher unemployment rate.


Looking at interest in the terms such as food stamps and part time jobs on Google, it is hard to talk about a recovery yet. Maybe U3 needs some criticism after all.