Fielmann AG, Controlling Analytics – Definition and implementation of an AWS Analytics Lakehouse architecture

The challenge:

The Central European market leader supplies both stationary and online trade via an omnichannel business model with digital sales channels. Distributed in different departments across the entire customer organization, different product teams work on the collection, evaluation and provision of data from the sales and eCommerce systems.

The Fielmann Group has set up a central data analytics team that makes various evaluations and dashboards within the customer organization easily accessible and usable. The aim is to enable the business to make data- and insight-driven decisions. Here, the “One-Stop-Shop” idea is pursued, making all relevant data available at a central location. Depending on the skill of the analysts, there are opportunities to prepare data themselves, carry out efficient ad-hoc evaluations, create management reports or gain new insights with machine learning.

The omni-channel business model and the associated digital transformation require an agile, scalable and high-performance analytics architecture. In addition to new data sources (data mesh architectures, streaming data, APIs), legacy applications (classic databases) can also be efficiently integrated online and offline as part of the mapping of the entire customer journey. Furthermore, the requirements for the availability of information from various stakeholders (controlling & finance, sales, marketing, logistics, etc.) must be mapped.

The solution:

Implementation of the solution architecture: “Analytics Lakehouse”

PROTOS Technologie GmbH supports the analytics area in the planning and automated provision of the AWS infrastructure for various ETL pipelines, the provision of ready-to-use data sources and exploratory data analyzes in order to optimally provide stakeholders with enriched data. PROTOS supports the migration of existing infrastructure components, data services, pipelines and data artifacts for the new architecture.

In order to implement processing with big data frameworks and modern database technologies sustainably and for a wide variety of data producers and consumers, previous analytical data pipelines are being migrated and from a data warehouse concept to an “analytics lakehouse architecture” based on AWS services (AWS S3, AWS Redshift, AWS Glue, AWS Lambda) rebuilt.

In addition, PROTOS supports with components for the simple, department-related and efficient handling of data, such as dbt for data provision and state-of-the-art reporting tools for visualization and ad hoc analysis.

Infrastructure Provision: “Automation”

The modernized infrastructure is provided with Infrastructure as Code (IaC), so it is consistent and under version control. is used here AWS CDK and hashicorp terraform. By using AWS managed services such as AWS Codebuild, the CICD infrastructure does not have to be manually maintained.

Automated deployments for each component, infrastructure customization, and ETL jobs are built using DevOps best practices (AWS Codebuild + Codepipeline, GitHub Actions). The high degree of automation and the potential of serverless/elastic cloud architectures is reinforced by the use of AWS Elastic Container Service (ECS), AWS Lambda combined with AWS Step Functions.

Implementation of data pipelines

In order for different data consumers to be able to work efficiently with the data, the focus is on high data quality by enriching it and bringing it into an analytics-compatible structure. PROTOS supports the Fielmann analytics team in particular in the development of data pipelines and ETL jobs. Spark is in action AWS Glue, AWS Redshift and in combination with AWS Lambda. This provides product teams with ready-to-consume data sources.

Further information

For more information on cloud, infrastructure-as-code, terraform, serverless and DevOps, also have a look at the PROTOS Technology blog.

Your PROTOS team