Designing Effective Data Organization
The rising complexity of data that the companies owned, stored, and collected gave rise to the data professional role in the organization. The companies may structure this role under the CTO, CIO role, or a new C role called CDO — the Chief Data Officer. The senior data role is the chief that designs the data organization to support the organization to be more data and insights-driven effectively. How was the data organization typically structured? In my experience and observation, there are three typical designs of data organization: Product-based data organization, service-based data organization, and strategy-based data organization.
Below, I elaborate what are the differences between those three organization structure, what is the pro and cons of each design.
Product-Based Data Organization
What type of organization usually employs this structure? Large and multinational companies typically use this design. The organization consists of a centralized team (usually sit in India) to build the data product and a decentralized team in the market/country that is responsible for deploying the data solution/product in each market.
A centralized team builds data products such as dashboards and AI products. The expectation is that one data product can be utilized by many countries, rather than expecting the country to build their own data product. The central team will employ business people and technical people. The business person will be a relatively senior person who understands the pain point that the business will face. They will formulate the product and get the feedback from the market. Once the business case is approved, they will build the product, and the market will bear the cost.
By leveraging this product, the data organization structure will be simpler and save costs. Imagine if a company has to build a data organization in each country; there will be a lot of costs to pay the data professionals’ salaries and infrastructure.
A decentralized team will sit in the market. This role will be similar to a business analyst/data analyst role. They need to understand the market’s requirements and propose a data product that suits them. Most of the time, they need to make sure that the data product uses the correct data, make sure that the output is correct and consistent, and build local data products required where the solution is not available from the central team.
Even though the organization is simpler and lean, many problems arise with its design; for example, a longer lead time to realize the requirement from the market because each solution needs to be discussed with another market, or a longer time to solve the issues due to different tickets raised by the different markets with different priorities. When the central team is unable to solve this problem, the market team needs to take over.
Service-Based Data Organization
The organizations that employ this structure are typically local organizations or start-ups. There will be a main central team consisting of a data engineer and data scientist and a decentralized data team that sits in each organization function.
The data engineering team will build the company’s data foundation and ensure the data is clean, correct, and accessible. Data scientists will support the creation of smart digital products such as recommendations or searches.
The decentralized team consists of data analysts who sit in each function. They will analyze the data, create a dashboard, and support with predictive analysis to support intelligent decision-making. In short, data analysts are responsible for insight and decision-making in each function.
This organization is more agile compared to the previous design. Problem can be solved quickly by data analyst. The side effect is a bloated data organization, which could cost the company a lot. However, if the business benefit is great, there is no problem in building this organization.
Corporate-Strategy-Based Data Organization
This organization is quite similar to a data-service organization. The difference is that there will be a central data team that will use data to solve more strategic business problems. This design is good for non-digital businesses, for example, CPG companies.
A high-caliber data team will sit with the corporate strategy unit to solve pressing issues such as marketing optimization, supply chain optimization, and many other strategic problems.
This could be a dream team for ambitious and intelligent data professional because they can leverage their skill to solve the company’s most pressing issues, with high expectations that might be quite stressful. For companies, this will be another cost to bear.
