Datama Assess - Our Models
Statistical models used by Datama to assess the significance are based on common models for a data scientist.
By default, Datama will select the most appropriate model based on the data you are comparing. It will likely be a Welch t test if you are comparing two discrete segments, and a forecasting approach if you are comparing two -long enough- periods.
However, you can select the test you want within model settings, under Test to Compute input (add link)
We have two main types of models:
AB Test tests:
Those are classic AB test significance calculation:
- Welch t-test using a frequentist approach (either for Bernoulli variables such has conversion or any other variables, such as basket value)
- Bayesian test
- Bootstrap test
In case you’re wondering which test is best to use for your AB test, you may want to report to this article which compares the accuracy and reliability of each test
Time Series tests:
Those are tests that you want to use when comparing two periods: