What is Synthetic Data?

What is Synthetic Data?

Synthetic Data is data that has been artificially created rather than collected from real-world events. It is generated by algorithms designed to reproduce the statistical properties, patterns, and relationships found in real datasets, without containing any actual records that could be traced back to real individuals, transactions, or events.

A simple analogy: if real data is a photograph of a person, synthetic data is a realistic painting of a person who does not exist. The painting captures the general characteristics, proportions, and features of a real person but represents no one in particular.

How Synthetic Data is Generated

There are several approaches to creating synthetic data:

Statistical Methods

Algorithms analyse the distributions, correlations, and patterns in a real dataset, then generate new data points that follow the same statistical properties. For example, if real customer data shows that 60 percent of customers are aged 25-40 and average order value correlates positively with age, the synthetic data will reflect these same patterns.

Generative AI Models

Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) learn the underlying structure of real data and generate new samples that are statistically similar. These models can produce highly realistic synthetic data for complex data types including tabular data, images, text, and time series.

Simulation-Based Generation

For specific domains, data can be generated through simulation. For example, synthetic driving data for autonomous vehicles can be created using 3D simulation environments. Synthetic financial data can be generated by simulating market conditions and transaction patterns.

Rule-Based Generation

Data is created according to predefined rules and distributions. This is the simplest approach, useful for testing and development when high statistical fidelity is not critical.

Why Synthetic Data Matters

Synthetic data addresses several critical challenges that organisations face:

Privacy and Compliance

Real customer data is subject to privacy regulations (Singapore PDPA, Thailand PDPA, Indonesia PDP Law, GDPR for European customers). Synthetic data that contains no real personal information can be used freely for development, testing, and analytics without triggering privacy obligations. This is especially valuable for companies operating across ASEAN markets with varying data protection requirements.

Data Scarcity

Many AI projects struggle because there is not enough real data to train models effectively. This is common for rare events (fraud, equipment failures, disease diagnosis) where real examples are limited. Synthetic data can augment real datasets by generating additional examples of rare but important scenarios.

Data Access and Sharing

Sharing real customer or operational data across teams, with vendors, or with research partners raises privacy and security concerns. Synthetic data enables collaboration without exposing sensitive information.

Bias Mitigation

Real-world datasets often contain biases that reflect historical inequalities. Synthetic data can be generated to be more balanced and representative, helping to train fairer AI models.

Testing and Development

Software developers need realistic data to test systems, but using production data in development environments creates security risks. Synthetic data provides realistic test data without the risk.

Synthetic Data in the Southeast Asian Context

Synthetic data is particularly relevant for businesses in ASEAN because:

• Regulatory complexity: Operating across multiple markets with different data protection laws makes synthetic data attractive as a way to enable cross-border analytics and AI development without navigating complex cross-border data transfer regulations.
• Emerging markets with limited data: Some ASEAN markets have less historical digital data available for AI training. Synthetic data can supplement limited real datasets to make AI projects viable.
• Financial inclusion: Fintech companies serving underbanked populations in Southeast Asia may lack the historical credit data needed to build lending models. Synthetic data can help bootstrap these models.
• Healthcare AI: Medical AI applications require large datasets that are difficult to assemble due to patient privacy. Synthetic medical data enables research and model development while protecting patient confidentiality.

Limitations and Risks

Synthetic data is powerful but not without limitations:

• Quality depends on real data: Synthetic data is only as good as the real data it is modelled on. If the source data is biased or incomplete, the synthetic data will inherit those problems.
• Edge cases may be missed: Synthetic data generators may not capture rare but important patterns that exist in real data. Models trained exclusively on synthetic data may fail on unusual real-world scenarios.
• Validation is essential: Synthetic data must be rigorously validated to ensure it accurately represents the statistical properties of real data. Metrics like distribution similarity, correlation preservation, and downstream model performance should be measured.
• Not a complete replacement: For most applications, synthetic data works best as a supplement to real data, not a complete replacement. Models trained on a mix of real and synthetic data typically outperform those trained on either alone.

Getting Started with Synthetic Data

1. Identify your use case: Determine whether privacy, data scarcity, testing needs, or data sharing is your primary motivation.
2. Assess your real data: You need at least some real data to generate high-quality synthetic data. The more representative the source data, the better the synthetic output.
3. Choose the right tool: Options range from open-source libraries (SDV, Gretel, Faker) to commercial platforms (Mostly AI, Hazy, Tonic.ai).
4. Validate rigorously: Compare synthetic data against real data using statistical tests and evaluate downstream model performance.
5. Start with non-critical applications: Use synthetic data for development and testing before relying on it for production AI models.