The Analytics Engineer and Scrum

In my previous blog in the series on the Analytics Engineer, I described what this role involves and how low-code solutions such as TimeXtender make this way of working possible. In this new instalment I look at how the Analytics Engineer fits within the Scrum way of working.

In software development, Scrum has replaced the waterfall method, but what does this mean for the information analyst? And how does it work in data development? In this blog I discuss the possibilities the Analytics Engineer offers!

From waterfall to Scrum

Traditionally, software was developed using a waterfall project method. The steps literally followed one another, just as a waterfall can consist of several levels. First the information analysis was done, then the technical design, then the actual development, then testing and finally going into production.

This all took far too long, and often people did not know in advance exactly what they wanted. Scrum solved this by developing a little at a time in short development cycles, with each delivery checked with the customer: are we on the right track? What is the next step? This delivers results faster and gives the client more opportunity to adjust course.

Scrum brought the team back together: everyone became a developer. That was a reaction against the specialist roles within the waterfall method. There, every step had its own specialist, so everyone had to wait for each other and information constantly had to be handed over.

Scrum also introduced user stories that were literally conversation starters: not a detailed elaboration of requirements by an analyst first, but a team of developers who immediately talk to the customer. What do you need? How does this help you move forward? The whole team thus did information analysis, no longer one person in one role.

The translation to data

Data engineers and front-end developers can also work in Scrum teams to jointly develop information products, such as reports or dashboards. They start with a user story, talk to the customer and work on solutions in short cycles, after which they gather feedback. In addition to engineering and development, they also do analysis. This is how the Analytics Engineer emerges.

The Analytics Engineer is someone who can carry out the entire data pipeline. The Analytics Engineer is thus occupied with:

• gathering the question from the business
• building the data structures
• and delivering the dashboards.

A separate information analyst is no longer needed. Information analysis has become a capability.

If we compare the waterfall method with Scrum:

Information analysis has become a capability of the Scrum team; separate analysts are no longer needed. The developers (front-end and back-end) become Analytics Engineers.

The Analytics Engineer as a solution

At E-mergo we see this working well in practice. In this way of working, Analytics Engineers are directly involved in the end product. This brings more involvement and better results. Engineers are no longer just a link in a long chain.

Not every engineer aspires to this combination of skills. In large organizations there is plenty of room for specialization, but for data teams of 3 to 10 people in particular, the Analytics Engineer combined with Scrum is a perfect solution