7 Golden Rules for Big Data Projects

Sunday April 27th, 2014

It seems every organisation has either jumped or is seriously contemplating jumping onto the Big Data bandwagon. In an industry where the hype is often followed by the despair, I feel somewhat ashamed that the IT Industry that I work in pushes the barrow often before the horse. As a result, organisations get hyped up around the advantages and the outcomes of the technology without putting in place the safeguards to ensure project success. Often when I talk to my customers about Big Data, I like to ask some basic questions to ensure that whatever we deliver is achievable and is not going to be just another piece of technology sitting on the shelf.

A survey by Infochimps.com asked IT professionals what are the challenges in a Big Data project and it delivered some interesting yet not to be unexpected results:

The following are some basic rules we should all be adopting if we are to embrace a Big Data Project:

  1. Don’t start a Big Data project without understanding the value– In my previous blog I talked about the complexities of putting a value on data from an asset point of view. But it should go without saying that when you go up to your CFO with cap in hand to kick start a Big Data Project, you should know exactly how to demonstrate a Return on Investment (ROI). Not only do you need to know intimately the ROI of your project, but be able to discuss in detail the benefits such as increased revenue. But don’t just say that revenue will increase by x%. You need to weave the benefits and tie them back to functionality. For example “The big data project will allow us to better understand our customers through the use of sentiment analytics. The results of which will allow us to respond quicker to customer’s needs resulting increased revenue”.
  2. Don’t ignore the wider Enterprise story– More often than not, we tend to pigeon hole Big Data projects into specific departments or functions. Big Data brings with it the power to draw upon data across an organisation and merge it together to create insights. HR, Sales and Customer data coming together to deliver value. Therefore don’t ignore the impact that your Big Data project will have across the organisation.  If you can tie in your ROI discussion in point 1 above to go beyond just your area by demonstrating value across the organisation, then the stronger your project will be and the more visibility it will receive.
  3. A big data project is not a technology project– A lot of the hype around Big Data is the products themselves and as such when we go to deliver a big data project it’s often about the technology. However a Big Data project should be seen as a business change project. Big Data changes the way we interact with our suppliers, customers and peers. Executive support from the CEO/CFO/CTO  is a must to ensure that the project garners the right focus and achieves the right organizational shift to make it a success.

    “A Big Data project is more about Business Change than the technology itself”.

  4. It pays to have the right type of Data Scientists onboard– The problem with the industry is that it seems that there are a lot more people calling themselves data scientists without any real qualification or experience. And most probably one of the reasons for this change is the higher pay grades that real data scientists demand because of their specialist skills. So for a project to be a success, you must build a strong delivery team which involves making sure you get a data scientist with the runs on the board. Don’t skimp lower and get a data/business analyst who now calls themselves a data scientist as you will get lower than expected results.
  5. Build a support  structure from the beginning– It’s very easy to get caught up in the moment and therefore overlook some of the less exciting components of a  Big Data project such as Data Quality, Data Governance and Metadata. However these components are often the building blocks to the success of the project. Get started early on these and results will come quicker and the output will be much more valuable to the end users.
  6. Build for tomorrow– The pace of change in this industry is extreme. New products, features and technology hit the market everyday. The problem this speed of change presents in a project is the issue of scope creep. If you design a Big Data solution with only the technology in mind today, then by the time it’s delivered it’s value will be diminished, or you’ll be making that many changes to the design during the project that it will never be finished. So be bold and become visionary. Understand the possibilities that big data can have on your organisation in the future and design a system that will deliver this.
  7. Involve the organisation. Often the vision of a Big Data project comes from highly paid consultants or someone from within the tech team. But Big Data is a pervasive technology and as such you should involve all parts of the organisation to contribute to the vision. Involve sales, marketing, HR and customer facing staff to bring ideas to the table. What do they want to see and be able to use to make their jobs easier? Not only may you get ideas not previously thought of, but subconsciously you are getting the organisation to embrace the concepts of Big Data as they have had the input.

Overall if we take note of the points above, we can really make big data projects a success and deliver on the hype that we all see and hear about.

Ben Davis is a Senior Consultant with Teradata ANZ. Based in Canberra he specialises in Government accounts and has over 15 years in the Information Management space including document/records management, web content management, database management and design and business case design. Ben holds a Degree in Law, a Masters in Business & Technology and is currently completing his PhD with research around data and cloud computing security. 

The following two tabs change content below.
avatar

Ben Davis

Senior Pre-Sales Consultant, Federal Government at Teradata
Ben Davis is responsible for pre-sales activities in the Federal Government market in Canberra. Ben consults across a broad range of government departments developing strategies to better manage the continual flood of data that these organisations are now facing. Ben is a firm believer that management of data is a continual process not a one off project that if managed correctly will deliver multiple benefits to organisations for strategic decision making. Previously Ben spent 6 years at IBM in a Senior Data Governance role for Software Group Australia & New Zealand and 10 years at Fujitsu in pre-sales and consulting roles. Ben holds a Degree in Law from Southern Cross University, a Masters in Business & Technology from the University of New South Wales and is currently studying for his PhD in Information Technology at Charles Sturt University. His thesis studies focus on data security, cloud computing and database encryption standards.
Category: Ben Davis Tags: , ,
avatar

About Ben Davis

Ben Davis is responsible for pre-sales activities in the Federal Government market in Canberra. Ben consults across a broad range of government departments developing strategies to better manage the continual flood of data that these organisations are now facing. Ben is a firm believer that management of data is a continual process not a one off project that if managed correctly will deliver multiple benefits to organisations for strategic decision making. Previously Ben spent 6 years at IBM in a Senior Data Governance role for Software Group Australia & New Zealand and 10 years at Fujitsu in pre-sales and consulting roles. Ben holds a Degree in Law from Southern Cross University, a Masters in Business & Technology from the University of New South Wales and is currently studying for his PhD in Information Technology at Charles Sturt University. His thesis studies focus on data security, cloud computing and database encryption standards.

4 thoughts on “7 Golden Rules for Big Data Projects

  1. avatarEmily Plachy

    Nice post. When I read the 5 big hurdles (talent, data, tools, time, platforms), I expected your 7 golden rules to be limited to talent, data, and technology. I was pleasantly surprised to see your rules encompass those to enable organizational change. In particular, rules 1, 3, 5, and 7, which address value, not a technology project, support structure, and involve organization.

    Reply
  2. avatarTushar Kale

    Good, practical, reasonable positioning and implementation points to be ponder before getting on Big Data wagon.

    Thank you Emily

    Reply
  3. avatarRichard

    I think finding the right talents is the biggest problem in the big data industry today. Though there are many IT professionals want to enter big data industry, the are not skilled enough. Companies are ready to pay high are skilled professionals as big data becomes the buzz everywhere. So first one should find the right talents to make their big data project a successful one

    Reply
  4. avatarIsrael Gaytan

    This is nice stuff!, thanks for sharing this. I am completly agree with this. I would like to argue that you have to understand business needs and data before you pick up the right technology for do the job.

    Regards

    Isra

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


*