Data Analysis

When designing the table structure for a small instant messaging scenario, since there is no need to consider the sharding design of the data model, and the business model and the statistical model can be directly integrated, for small business scenarios, you can quickly realize unpacking through code Ready-to-use and efficient basic data analysis functions, and extended to provide common statistical APIs based on index fields.

However, the Turms project is designed for medium and large instant messaging scenarios. Data analysis and business implementation must be separated at the architectural level, which also includes the separation of business models and data models. If you need to perform data analysis, you can collect the measurement or buried point logs generated by the turms-gateway and turms-service servers, and use cloud services or self-developed implementations to analyze them.

In addition, considering that there are indeed many common and common IM-related statistical data, we will open a new project turms-data to be responsible for data analysis, and cooperate with Turms server and turms-admin to realize: log and database data collection , Data warehouse construction, analysis and statistics of business indicators, result visualization and other functions.

Note: Since Turms was mainly designed for small instant messaging scenarios in the early days, all API query fields were implemented based on indexes at that time, which could ensure query efficiency. But later turned to design for medium and large scenarios, and many indexes were removed, but the query fields of the corresponding APIs (especially the statistics API) were not removed, so there are still some APIs (especially the statistics API) The implementation of query parameters will use full table scan, which is Legacy code. We will classify these APIs according to the implementation performance to ensure that some inefficient APIs will not be misused.