On-premise, Cloud, or Hybrid? Discover the Options for Your Data Infrastructure with TimeXtender

There are many considerations to make when it comes to your data infrastructure. Do you go fully to the cloud? Do you want a hybrid setup, or is it important that data processing takes place on-premise on physical servers? Depending on your company's goals and needs, it's essential to think about tooling that can support your choice. For our clients, across all three infrastructure options, we like to work with the TimeXtender data management platform, because it offers options for all three. It also offers ways to move from on-premise or hybrid to the cloud. Curious about the possibilities for a data infrastructure with TimeXtender? Read on.

TimeXtender

TimeXtender is a Data Warehouse Automation solution that lets you develop your data foundation in an automated way. TimeXtender orchestrates your infrastructure from the source systems all the way through to your data products. This means all your data is collected and processed in one central place. This makes it easier to safeguard a single version of the truth and use this data in data products such as apps, portals, and analytics dashboards in, for example, Qlik or Power BI. Because TimeXtender can adapt to the resources you have available, there are many possibilities, and therefore also many considerations. Below are some situations and recommendations described.

Which infrastructure fits us?

Once you've become enthusiastic about the possibilities for managing your Data Platform with TimeXtender, you'll quickly arrive at the question of what kind of infrastructure fits with it. TimeXtender can work with architectures ranging from fully on-premise, to hybrid, to fully cloud-based.

Figure 1 Infrastructure choices per layer

Because TimeXtender tells you which resources should be used to store, transport, and manipulate data, you can also adjust this along the way and thereby migrate from one infrastructure to another. This makes it easy to move your Data Platform from an on-premise installation to the cloud, for example.

The question of which setup to use therefore becomes much less of a big deal: it's much easier to make a change afterward in the choice of components and location. This means TimeXtender can adapt to the way your organization works, instead of the other way around.

On-premise

When all your company infrastructure runs on-premise, it may make sense to set up your Data Platform there as well. In that case, you'd choose to store your Data Lake and Data Warehouse layers in a SQL Server instance on a virtual machine, and in a compact setup you can add TimeXtender alongside it. Depending on the BI solutions you want to support, you're then already done. For optimal support with Power BI, it's a good idea to use SQL Server Analysis Services to house Data Marts; Tableau and Qlik don't require any extra infrastructure beyond their own system.

Hybrid

Perhaps you have your source systems running on-premise, but you want to run your Data Platform in the cloud, for example so as not to burden your IT department with resource management, or because you want to be able to use Platform as a Service (PaaS) services. In that case, you can use a hybrid architecture: sources on-premise, Data Platform in, for example, Azure. The only important consequence is that you need to take into account the bandwidth of the network connection between your on-premise network and Azure, which will usually be more limited than when you run your Data Platform on-premise in the same network as your source systems. Another example is to keep all data up to and including your Modern Data Warehouse on-premise and bring your semantic endpoints to the cloud.

Lift-and-shift

If the organization starts on-premise and later adopts a cloud policy, you point TimeXtender to the new components and switch over easily. This way, you lift-and-shift your data and infrastructure to the cloud. In general, you can also use TimeXtender as a tool to migrate databases from on-premise to the cloud.

Cloud

Because TimeXtender uses the Microsoft Data Stack, it integrates very well with the PaaS services Microsoft offers in Azure. Of course, you can also obtain components such as SQL Server from other cloud providers. Depending on the number of sources being unlocked, the need to support Data Science and Advanced Analytics, and the performance you want to deliver, you can make different choices between the main elements of a cloud infrastructure: Azure Data Factory and Azure Data Lake make it possible to unlock many more sources in parallel, Azure Synapse Analytics makes it possible to query gigantic tables more efficiently and integrates well with many tools focused on Advanced Analytics, and Azure SQL Hyperscale offers a great balance between cost and storage size.

If your organization already has a foothold in Azure, it's easy to set up TimeXtender there as well. The Azure Marketplace templates make it even easier to get started.
Want to read more about Azure in combination with TimeXtender (and Power BI)? Read this blog.

The next step

Once you have an idea of the main components you want to include in your architecture, it's time to sketch out a more detailed setup and try out what TimeXtender brings you. As Partner of the Year, E-mergo is the ideal expert to assist you in taking the next step in the field of Data Warehouse Automation!

Written by Ruairidh Smith,
Senior Consultant at E-mergo