Introduction #
SmartPip (Data Platform Task Scheduling Module) is SmartChart’s enterprise-grade data scheduling center, supporting visual DAG workflow orchestration, scheduled task execution, and data quality monitoring. Built on Apache Airflow, it provides rich ETL drivers for data extraction, transformation, and loading.
Core Features #
| Feature | Description |
|---|---|
| DAG Visual Orchestration | Drag-and-drop task dependency design |
| Rich Data Sources | MySQL, Hive, Impala, Oracle, Kafka, DataX, etc. |
| Scheduled Execution | Flexible Cron expression configuration |
| Data Quality | Built-in data validation and alerting |
| Cross-DAG Dependencies | Support for inter-DAG task dependencies |
