Getting Started: Data Sources
Every synthetic dataset starts from a description of the statistical structure you want to reproduce. The module accepts that description from three different sources.
| Source type | Use when | What you provide |
|---|---|---|
| Raw data | You have the original dataset | An Excel file of observations |
| Descriptive statistics | You only have a published summary | A table of per-variable statistics (mean, SD, min, max, distribution, categories) |
| Manual input | You want to define variables and parameters by hand | Variable definitions entered through the interface |
The default source type is raw data. Whichever route you choose, the module converts your input into the same internal representation before generation begins.
Raw data upload
This is the most common path: upload your real dataset as an Excel file, and the module analyses it automatically.
Accepted files
| Property | Requirement |
|---|---|
| Format | Excel .xlsx or .xls (HTML table exports .html/.htm also accepted) |
| Maximum file size | 50 MB |
| Layout | One row per observation, one column per variable, with a header row |
The Upload Your Dataset panel accepts a file by drag-and-drop or by browsing. Once dropped, it is validated immediately.
File validation
Errors (block the upload):
| Condition | Message |
|---|---|
| File larger than 50 MB | "File size exceeds 50MB limit. For larger datasets, please contact us at info@epicosai.com for custom solutions." |
| Not a recognised format | "Invalid file format. Please upload an Excel file (.xlsx or .xls)." |
| File cannot be read | "Failed to read the Excel file. Please check the file format." |
Warnings (do not block, but flag a concern):
| Condition | Warning |
|---|---|
| More than 200 columns | "Dataset contains N columns. Maximum recommended is 200. Processing may be slower." |
| Fewer than 2 columns | "Dataset has very few columns. Make sure your data is properly formatted." |
| Fewer than 10 rows | "Dataset has fewer than 10 rows. Results may not be statistically meaningful." |
| More than 50,000 rows | "Dataset contains N rows. Processing may take longer than usual." |
| Empty/blank column headers | "Some column headers are empty or contain only spaces." |
| Numeric column names | "Some columns have numeric names. Consider using descriptive text headers." |
After a successful upload, the module reports the dataset's dimensions and moves on to automatic analysis.
Descriptive statistics
When you do not have (or cannot share) the raw data, you can drive generation from a table of summary statistics -- exactly the kind of "Table 1" that appears in a published paper.
Accepted input
The statistics table can be provided as an Excel file or an HTML table (e.g. copied from a paper or webpage). Each row describes one variable.
Recognised columns
The parser is flexible about column names -- it recognises common synonyms:
| Statistic | Accepted column names |
|---|---|
| Variable name | variable, variablename, name, var, column |
| Mean | mean, average, avg, mu |
| Standard deviation | std, stddev, sd, standarddeviation, sigma |
| Minimum | min, minimum |
| Maximum | max, maximum |
| Distribution | distribution, dist, distributiontype |
| Categories | categories, levels, values |
| Probabilities | (category probabilities, where applicable) |
If no distribution is specified, the variable defaults to normal; if categories are provided, it is treated as categorical.
Validation messages
| Condition | Message |
|---|---|
| No usable statistics found | "No statistics data found. Please ensure your file contains variable names, statistical measures, and proper column headers." |
| Table cannot be parsed | "Unable to parse the statistics table. Please ensure your file format matches the expected structure." |
| HTML table missing rows | "Table must have at least a header row and one data row." |
This route lets you generate a dataset matching a paper's reported statistics without ever accessing the original records -- useful for replicating published analyses, building teaching datasets, or prototyping a study design.
Manual input (template)
For full control, define everything by hand using a downloadable Excel template. This is the route for designing a dataset from scratch.
The template
The module generates a structured Excel template (synthetic_data_template.xlsx) with multiple sheets:
| Sheet | What you fill in |
|---|---|
| Variables | The list of variables and their types |
| Parameters | Distribution parameters for each variable (mean, SD, min, max, etc.) |
| Correlations | The correlation matrix between variables |
Rows whose names contain "example" are treated as illustrative and ignored during parsing.
Filling it in
Define each variable's type and distribution parameters directly, and specify the correlation matrix in the Correlations sheet. When you upload the completed template, the module parses all three sheets and assembles the full configuration.
What happens after the source is loaded
Regardless of which source you used, the module produces a unified internal model:
- A list of variables, each with a detected or specified type
- A distribution with parameters for each variable
- The relationships between variables (correlations and effect sizes)
You then review and adjust these in the Configure Settings step before generating.