Organisations have a life of their own. Sometimes they expand and at other times they contract, much as how the lungs expand and contract with breathing that provides life to us! However, barring the times when I perform Pranayama to control my breathing, most of the expansion and contraction of my lungs is an involuntary mechanism!

On the other hand, organisations cannot be left to themselves to expand and grow if they are to be a going concern. For survival, they need to be managed effectively with right decisions taken at the right time.

This is where an organisation’s vision and strategy helps in establishing its structure, putting an end to the long-held chicken and egg debate of which came first!

Span of Control is a mechanism that is used to structure the organisation for effective decision making. Narrow span of control leads to tall organisational structure, while wide span of control means flat organisation structure which translates to inverse relationship between span and levels of the organisation. Centralised and decentralised decision making styles are the result an organisation structure.

Henry Mintzberg probably provides a more eloquent description of organisation structures in his book Structure in Fives. The aspect of the organisation structure that interests me most is Adhocracy. Why? Hold your breath for a moment, I will explain shortly!

So, what has this got to do with data warehousing? It is my theory that the data warehousing architectures are closely aligned with the organisation’s ever changing strategies and structures and therefore, will evolve to support more effective decision making.

Take data stores and data marts as examples which have had humble beginnings to support the departmental/divisional strategies but an enterprise data warehouse (EDW) architecture enables enterprise level decisions. Rob Armstrong’s whitepaper is a great source of information about data stores, data mart and the benefits of enterprise data warehouse (EDW).

I am already seeing an evolution of data warehousing towards what the Gartner Group calls Logical Data Warehouse (LDW). We can see why!

Web 2.0 and Telco 2.0 are the trend that is driving the need for this kind of change which is translating to the industry jargon of “Big Data”. This is where I believe Henry Mintzberg’s Adhocracy comes into play. The Network organisational structure is a form of Adhocracy that supports a number of Web 2.0 and Telco 2.0 initiatives such as crowd-sourcing, social networking, social media, online business partners and so forth. No doubt, this evolving trend and diversity of needs will require a different data warehousing architecture to support it.


Teradata is at the fore-front of this trend in data warehousing and addresses these needs with Unity - the unified ecosystem for integrated data warehousing enabling the diverse requirements of the evolving organisational ecosystem.

Learn more about Teradata at the “Ask the Experts” session during Teradata Universe. See you there soon!
 

Over Easter I was driving through Geelong when one of my daughters started chirping “Daddy, that iPad is just like your one”. Looking around to see what she was talking about, my eye was drawn to a picture of an iPad on the back of a rubbish truck. On closer inspection it would appear that the local waste management company had decided to direct some of their marketing budget into setting up a Facebook fan page, a Twitter and a Linkedin account; and if you reached out to them then you might win an iPad. Given I had just bought a new iPad, the incentive of getting another one was not all that big, but I was very interested to understand if indeed people would want to be a fan of a waste management company.

So that night I went and had a look to find that the Twitter and Linkedin accounts had not generated any chatter. But the Facebook page had been a little more successful, they had pictures of the team, pictures of their new truck (coincidentally the same one that I had seen), and that is where the experiment seemed to stray from the original hope. The next 10 or so posts on their wall focused on a crude dialog of comments regarding their team and their service. While this was a little amusing to me, I am sure that the Head of Marketing (assuming that such a grand title exists, it was a small team) never envisioned that their venture into the world of online marketing would go so wrong.

The journey into social media as a tool set for organisational marketing is still in its infancy for most companies, and I am sure that the example that I have been talking about above is not that far from some experiences that other organisations have had. The amount of chatter on the waste management fan page was small, so being able to turn negative sentiment into positive outcomes would not be too difficult, but many organisations out there will have more passion associated with their brand. And with more passion comes more chatter. Looking at some of the larger Australian brands, the amount of positive chatter to negative chatter can become over whelming if the strategy is to turn negative sentiment into positive outcomes.

The introduction of big data platforms and the associated analytical tool sets has finally given us a means to addressing the torrent of chatter. Being able to understand negative sentiment inside the social channels and then to track how that negative sentiment plays out across the entire customer experience is now possible. With the understanding that is derived from the analytics, organisations can now focus on how to drive positive outcomes.

I have often heard the phrase, “What are the three things that keep the CEO awake at night?”. These days, I believe there are three letters that keep CEOs of most major industries awake: OTT (Over The Top).

OTT players are behind the major force that is redefining the industries which is rapidly shaping customers’ behaviours now. Let’s see how:

The ATM has been the window to bankers’ customers; for the retailers, it has been the POS terminals; whilst the TV has been the window to the broadcasters and Pay TV providers’ customers. Let’s visualise a scenario wherein the CEOs of all these and other industries are staring out their office windows pondering about the future while their customers walk right into the retail store, take their smartphones out of their pocket, scan the bar code of the item that is meticulously stacked up on shelf and compare prices of the item on eBay, watch a YouTube demo of the product, decide to buy the item 5% cheaper at eBay and pay for it through PayPal - all within the comforts of the brick-and-mortar shop – and end up walking out of the store feeling happy about their frictionless Over The Top shopping experience! Not to be left out, the CEO of the telecom gazes at the balance sheet watching the top line and bottom line dwindle while the OTT players are gaining their customers’ mindshare, leaving the telco as a mere “dumb pipe” carrier!

Sounds grim! Yes, much of it is real but a lot can be done by these businesses using analytics to understand consumer behaviour and enhance the customer experience to retain them. You only need to look at eBay’s Red Laser, Amazon’s Price Check, Square’s non-merchant system payment and how Venmo enables splitting a restaurant bill at social dining occasion and peer-to-peer money transfer that leverages the Facebook friends-circle, are all shaping the future and shaking the industry ecosystem. All this happens on a single window to the world of commerce that is in the hand of the consumer – the Smartphone!

It is probably unthinkable for 100+ year-old companies such as Kodak, the Apple and Google of its time, to be at the verge of collapsing under its own weight. In 1975, Kodak invented the digital camera, the same year that Theodore Levitt wrote his famous article Marketing Myopia in the Harvard Business Review, but Kodak executives did not appear to have read it! It’s a pity, Kodak did not see the success of its own innovation, left it for its competitors to claim the glory (possibly because Kodak thought it was still in the chemical business and failed to see the shape of things to come!). Kodak was not alone; Polaroid, Borders, Blockbuster et al. have been recent victims of their own success!

Theodore Levitt talks of the fateful times for several industries leading up to the 1970s (i.e. railroad, electric utility, grocery store chains, the Hollywood et al.). What is interesting is that history seems to repeat itself, but only the players are different now!

Several industries are at cross-roads now, more than ever before. What business is the telco in? Are they in communications or media or retail or banking? For that matter, what business are the brick-and-mortar retailers in? More importantly, what signals are they getting from their consumers about changing behaviours? Are these signals falling on deaf ears or going over the top? There is always someone else who is listening to the signals attentively and taking timely action. This time around it will be the OTT players who are paying attention and taking charge!

 

 

Businesses have a choice, if they want to overcome the insomnia. Looking into the crystal ball is an option, but that is sure to miss the fingerprints that your customers leave behind with every interaction with your company. Which way you go depends on your ability to connect the dots! Looking at what happened is useful, but analysing why it happened and more importantly, why it did not happen compared to industry norms will provide greater foresight! Checking the rear-view mirrors often helps notice the blind spots as well as to attain a sense of the frustration your customers may be experiencing during the long bumpy ride with your business and may help to prevent them from jumping off the bus at cross-roads!

Some in the industry deny existence of a problem / trend among their customers. I like Theodore Levitt’s quote, “If thinking is an intellectual response to a problem, the absence of a problem leads to absence of thinking”. Analytics helps to uncover problems before they surface. Analytics is not only intellectually stimulating but also promises to see future obstacles on the road for navigating a safer driving experience – or even for survival! 

 
Sundara Raman  

As I covered in my last blog entry, “Where did my capacity go?” the fact that there is increased capacity for processing work leads to more work being done. So how does one accurately forecast the projected capacity requirements when all of the facts aren’t readily available?

Simply put, you need to get in tune with the consumers. What are the key business opportunities they are pursuing? What has been their experience to date? What is on their wish list that they just don’t get to?

In general terms, I have found that a good, integrated data warehouse can be compared to an iceberg. On the surface you see about 20% of the work. Hidden below is all of the other work which could be done, but isn’t being done on the platform for various reasons. This could be as simple as just asking that next query once you get a set of data, pulling a bunch of data and performing analysis in Excel, or even taking a much larger extract and dumping it into an external mart.

In one example, we used the system metrics to show the impact of a major upgrade. After adding approximately 20% more capacity to the platform, we anticipated improved response times and less governance with workload management. However, the initial impression was that the upgrade did not happen! A deeper analysis revealed that during critical business hours on a Monday, the system serviced around 350,000 queries. The Monday after the upgrade this leaped to 415,000 queries! Of course, the initial impression was that the upgrade did not relieve capacity constraints. This extra demand was not anticipated or considered when sizing the upgrade. This led to increased governance, and an earlier upgrade than originally forecasted.

Opening a dialogue with the community can lead to a better holistic solution. The business unit which is offloading data may be able to move that process to a data lab within the data warehouse reducing latency and improving results. The user, who is putting data into Excel for analysis, might just need one metric added to their reports and that can be leveraged across all users. Additional demand isn’t necessarily bad, as long as it drives additional business value. Not anticipating this demand can lead to frustrations as the “new capacity” is quickly consumed.

Once you gain insight into the usage, or the lack thereof, you can start to see the pent up demand. What new value could the Consumer Insights group drive if they could drill deeper into their customer segmentation? What benefits could the Store Operation’s team find if they could bring more variables into their cost model? Once these are quantified, they can be forecasted in to a post-upgrade consumption model. This can take the surprises out of the upgrade and get a much happier user community!

John Berg

A common question I have been asked by customers over the years after they have upgraded their systems is, “Where did my capacity go?” This can be a difficult conversation to have. As I was researching other items, I came across a Wikipedia entry on Jevons’s Paradox. It states, “In economics, the Jevons paradox (sometimes Jevons effect) is the proposition that technological progress that increases the efficiency with which a resource is used, tends to increase (rather than decrease) the rate of consumption of that resource.” It got me thinking that this could be applied to this situation.

When we conduct Capacity Planning on a Teradata system, we look at workloads based on defined user groups, and their historical consumption rates. We tend to see organic growth rates from about 2% - 5% on a monthly basis. This includes both an increase in the underlying data volumes, as well as new user adoption of these capabilities. We also forecast expected new workload being introduced and it’s take up rate, and growth. These are the primary contributors to understanding capacity on Teradata and are captured, measured and reported on through the Capacity Planning and Performance Tool. The missing, and much harder factor to measure is pent up demand.

This also leads to another Wikipedia entry: Parkinson's Law “Work expands so as to fill the time available for its completion.” Or further generalised as, “The demand upon a resource tends to expand to match the supply of the resource.”

Let’s say you are an analyst in an organisation. One way or the other, you will be putting in your 8 (or 10) hour day looking for the next great insight in your business. Before the upgrade to the system, you are able to run 3-4 analytic queries a day, review the information, revise and refine, and work your way towards the answer you were seeking. Then you come in one Monday morning, and see your queries finishing much quicker than before. Yes, the system has been upgraded with more capacity, and that extra capacity results in more throughput. So now, instead of running 3-4 analytic jobs, you can increase your work to 6-8 jobs per day. The more questions you ask, and the more answers you get, lead you on and on down the path.

Applying Jevon’s Paradox enhanced by Parkinson’s Law – upgrading your system will only serve to increase demand as the efficiency improves and more work will be done in the same amount of time.

So, where did your capacity go? It went to the users!

Next up… Part 2: Addressing Demand and Capacity

John Berg