Blogs-Tapdata-Tapdata

Release Notes

Tapdata Cloud 3.5.6 Release Notes

Feature Optimization ① Create a unique index for non-primary key update conditions to resolve data duplication issues. ② Optimized data source connectivity, adding SSL connection support for some data sources ③ Optimize user interface interactions. Issue Fixes ① Fixed an error occurring during synchronization when the table name contains a period. ② Fixed an issue where error messages did not include the table name when there was an exception during task data processing. ③ Fixed an issue where, when specifying an Agent for a task, it still showed a task quota reached error.

Nov 10,2023

Release Notes

Tapdata Cloud 3.5.5 Release Notes

New Features ① Added support for creating distributed tables when using MongoDB as a target. ② Added support for disabling node functionality in tasks. ③ Data transformation tasks now include an ‘unwind’ processing node. Feature Enhancements ① Enhanced table selection for copying tasks with quick filtering for tables with and without primary keys. Added support for non-primary key tables with unique indexes. ② Unified and optimized handling of abnormal display issues for operation buttons in the engine interface. ③ Added a DEMO data source in the onboarding process, enabling users to use it directly. Issue Fixes ① Fixed the issue where MongoDB as a target would result in an error during insert operations when no shard key was present. ② Fixed the problem with MongoDB’s lack of support for ‘replace,’ making deleted fields undeletable.

Nov 10,2023

MongoDB Tutorial

A Convenient Way to Build Real-Time Data Model in MongoDB

Introduction Unknowingly, we’ve arrived in the year 2023, and MongoDB has already released version 7.0. MongoDB has pioneered the unique category of document model databases, bringing us many surprises in terms of products, technology, and business. More and more developers are developing their applications with MongoDB. Often times, when developers are working with MongoDB, they need to move data from existing relational databases into MongoDB to support new use cases. Because MongoDB’s unique, denormalized document model, it often means we need to sync data from multiple relational tables into a target JSON. This article is aimed to illustrate a convenient method to achieve this purpose. MongoDB Unique Features Before the advent of MongoDB, developers primarily working with SQL databases often followed the traditional three-normal form standards when designing data table structures. Under this standard, data from different domains that were related to each other were often designed as separate table structures and then linked using some association criteria. While maintaining data accuracy was relatively easy due to the existence of independent data, developers often had to write complex SQL queries to meet their business requirements. MongoDB’s usage pattern, on the other hand, is quite the opposite. MongoDB encourages developers to...

Nov 02,2023

Application scenarios

Build A Real Time Data Platform with MongoDB and Tapdata

In the age of big data, almost every business feels the urge to establish its data platform. Presently, there are numerous offline solutions available, including those within the Hadoop ecosystem like CDH and TDH, along with traditional data warehouses. Yet, two substantial hurdles hinder enterprises: the need for “real-time” data and the imperative of “data fusion.” On one hand, with the constant evolution of IT architecture and the advent of omnichannel marketing on the business front, the demand for real-time data has surged. On the other hand, the legacy of decades of digitalization has resulted in a landscape of data silos and isolated systems. Only when these data islands are fused can they be effectively utilized. So, what’s the solution for crafting an enterprise-level real-time data fusion platform? Tapdata has unearthed a promising approach. This article will exemplify the process using one of Tapdata’s clients in the retail sector as a case study. We’ll demonstrate how to swiftly construct an enterprise-class real-time data fusion platform, harnessing the power of MongoDB. Tapdata’s client is a distinguished player in the jewelry retail industry, boasting a rich heritage dating back to the 1990s. With a presence in over 700 retail outlets across mainland...

Oct 12,2023

Release Notes

Tapdata Cloud 3.5.4 Release Notes

New Features ① Added the feature to build materialized views. ② Added unset support for MongoDB to MongoDB synchronization. ③ Added support for retrieving read-only access information for subscribed Atlas databases. Feature Enhancements ① During the task setup process, added instructions for the public IP address of fully managed Agents when establishing connections. ② Improved the field filtering experience for data validation functionality. ③ Added support for searching nodes on the DAG page. Issue Fixes ① Fixed an issue where restarting the Agent would log the operation as an Agent upgrade.

Oct 11,2023

Release Notes

Tapdata Cloud 3.5.2 Release Notes

New Features ① Added Python Processor Node, allowing users to process data using Python scripts (requires upgrading to the latest engine version). ② Introduced a “Contact Us” feature for easy access to Tapdata support when users encounter issues. ③ Added support for Redis-to-Redis synchronization capability. Feature Enhancements: ① Enhanced error code functionality for data sources. ② When configuring email alert notifications and no email is bound, users will be guided to complete the email binding process. ③ Improved notifications for when the maximum task limit is reached, providing a quick access option to guide users in upgrading.

Oct 11,2023

Release Notes

Tapdata Cloud 3.5.1 Release Notes

New Features ① Added new options for Atlas storage when ordering on the international site: Free trial, M10, M20, M30. ② Added support for adding existing Mongo storage when ordering storage on the international site. ③ Introduced a new feature: Master-Slave Merge Nodes. Functional Enhancements ① The documentation for configuring data sources has been updated to include embedded online documentation. ② Provided parameter guidance for connecting to core data sources during connection creation. Resolved Issues: ① Resolved an issue where previously executed tasks were unable to access the run monitoring page.

Sep 08,2023

Notice

Release Notes

Tapdata Cloud 3.4 Release Notes

New Features: ① Guided task creation based on the Data Panel. ② Added a new Dashboard page. ③ Added support for DWS data source as a target. ④ Added an initialization guide for the Real-time Data Center. ⑤ Added domestic B2B payment method. Feature Enhancements: ① Integration of data replication and Data Panel. ② Optimized payment confirmation page. ③ Improved and adjusted the process for creating calculation engines. ④ Enhanced data validation usability. ⑤ Implemented error codes for MongoDB data source. ⑥ Incremental delay indicator on the left side of the runtime monitoring page now uses the Y-axis for task incremental delay alert threshold. ⑦ Enhanced new user onboarding process, added offline deployment guide. ⑧ Improved presentation of sample data. Resolved Issues: ① Fixed the issue where using custom SQL prompts unsupported MongoDB as a target. ② Fixed the problem where if the task’s automatic reset fails after enabling automatic periodic scheduling, it won’t attempt to reset again in the next running cycle and the task won’t be scheduled for execution.

Sep 08,2023

Notice