Data integration for low-code apps

A data-driven organisation starts with a solid foundation. This foundation consists of the right tools and knowledge to process data quickly and make it available to users. Data integration and data management have always been important for Data Analytics, so that the right data is available in the right form at the right time to make important decisions. Today, however, being a data-driven organisation means more than clear reports and smart dashboards. With low-code software development, organisations can digitalise business processes and bring new services to market in a short time, using innovative, purpose-built apps. For many companies, far-reaching digital transformation is essential to their survival, and it starts with a solid data foundation. In this blog, you'll read why data integration is so important for the success of low-code applications and how you can go about it.

Low-code software development

Low-code software development has grown enormously in popularity in recent years. The market for low-code platforms has existed for quite some time, and the tools have now matured into fully fledged products. With low-code, new web applications and mobile apps can be delivered much faster (in weeks rather than months) than with traditional development methods. One reason is that the apps are developed visually, using building blocks and drag-and-drop functionality to model user interfaces, functionality and workflows instead of coding them by hand.

Low-code platforms take much of the complexity of software development off the hands of the developers who work with them. As a result, in many cases you don't need any programming knowledge to get started with low-code and develop applications (or components of them). This gives rise to a new generation of developers, also known as citizen developers. By working with a low-code platform, an organisation needs far less IT support for developing, testing, deploying and monitoring applications. The looming shortage of IT staff is a driving force behind the rapidly growing adoption of low-code.

Leading research institutes also recognise the power of low-code platforms. Gartner, for example, expects that within two years more than half of all large organisations will be using low-code development tools to build applications, and that the number of citizen developers involved in app development will be four times greater than the number of professional programmers.

It is clear that organisations can achieve their digitalisation and innovation goals with the help of low-code. Low-code software development is the way to have developers and business users work together to build innovative solutions faster. This removes a major bottleneck for innovation: the availability of IT staff. But there is another important point of attention that can stand in the way of the success and speed of digital transformation: the data.

The importance of data integration

Data plays a crucial role in every digital innovation. From experience, we know that part of the time we spend on application development is lost to data preparation. The data an application needs sometimes comes from different systems and must first be identified, analysed, interpreted, integrated and transformed before it can be used in an application. The time spent on this can amount to as much as 50 per cent.

A modern data management platform can provide the solution. By bringing data from different source systems together in a data warehouse in a consistent, automated way, the data can be managed from a central location and made available to users and applications. We know from the past that developing and maintaining a data warehouse takes a lot of time and money, but with modern data warehouse automation tools such as TimeXtender and Qlik Data Integration, this is no longer the case. These tools let you model the data flows in a graphical interface, while the databases, tables and ETL scripts are generated automatically. On top of that, metadata and data lineage (the origin of data) are automatically visible, and documentation can be generated from the tools. This makes it possible to dramatically speed up the development of a data warehouse and improve its quality.

With data warehouse automation tools, new sources can be unlocked quickly and datasets made available fast. Application developers can then get to work with the data straight away, because they know where the data is, because the data is available in the right form, and because it comes with the right metadata to understand its content.

At E-mergo, we have put the combination of low-code and data warehouse automation into practice in developing the E-mergo Customer Portal, a web application we recently built with Mendix. In the customer portal, E-mergo customers and employees have access to personalised information. For a customer, this means they have direct insight in the portal into ongoing projects, invoices and open tickets they have logged with the support department. Mendix retrieves the information in the portal from E-mergo's data warehouse, which was developed with TimeXtender. TimeXtender ensures that the relevant datasets from various databases and SaaS applications are combined in the data warehouse.

Data integration in low-code platforms

E-mergo develops high-impact low-code solutions with Mendix and Power Apps. The vendors of these tools have now also recognised the importance of data integration. We see this reflected in new features recently added to Mendix and Power Apps that focus on streamlining data integration and reusing datasets.

Mendix Data Hub

Last year, at the Mendix World 2019 conference, Mendix introduced the Data Hub. This is an add-on module that adds a kind of data virtualisation layer to the development platform. In the Data Hub Catalog, developers can search for datasets, explore them and retrieve metadata. The datasets in the catalogue are OData services that can easily be added while developing new apps in Mendix Studio Pro. New datasets can be added to the catalogue automatically as soon as a developer creates an entity in an app. Datasets can also be added to the catalogue manually, and there is a Data Hub API. This allows new services to be registered in the catalogue for datasets from source systems that are not yet used in Mendix. In addition to finding, registering and consuming datasets, Mendix Data Hub offers various management features for implementing data management. Think of setting up data governance, analysing data lineage and monitoring data quality. The Mendix Data Hub makes it easier to share datasets between applications, which in turn speeds up the development of new applications.

Mendix Data Hub

Microsoft Dataverse

Last year, Microsoft introduced Dataverse for Power Apps. Dataverse is the new name for Common Data Service, the data storage layer for Microsoft's Power Apps and Dynamics 365. It is a SaaS storage service in the Azure cloud in which the datasets for low-code apps can be stored. First of all, Dataverse contains a base set of standard tables in which business information from Dynamics 365 is stored. In addition, Dataverse offers various connectors and functions for creating your own tables by automatically loading and combining datasets from different file formats and source systems. Just like the Mendix Data Hub, Dataverse lets you set up a governance structure to secure datasets while at the same time promoting the sharing of datasets between applications.

Microsoft Dataverse for the Power Platform

Conclusion

Low-code platforms have developed into very mature products with functionality that goes beyond just building apps. Data integration and data management are receiving more and more attention within the tools. That is an important development, because a solid data foundation is essential to deliver on the promise of low-code. Without this foundation, innovation projects cannot be delivered on time and within budget. For organisations with digitalisation ambitions, it is therefore important to focus on the data foundation. After that, application development can begin.

Want to know more about E-mergo's high-impact solutions for low-code apps and data warehouse automation? Then get in touch with one of our account managers!