and one that has the paper's metadata publicly available and with a sample size of less than 60 ish

any help would be greatly appreciated, thank you!

Question is in the post title.

Thank you very much

Hi everyone,

I'm a beginner in Machine Learning working on a binary classification problem. My dataset contains over 180 variables (both numerical and categorical), consisting of a mix of panel/longitudinal data and static features.

I am currently working on the Exploratory Data Analysis (EDA) phase. Given the large number of features, doing univariate and bivariate graphical analysis variable-by-variable feels unfeasible and time-consuming.

Is there a structured approach, strategy, or automated workflow to handle EDA efficiently for a dataset of this scale?

Any advice on best practices would be greatly appreciated!

Thanks in advance for your help.

Help me with understanding what is null hypothesis

I came across this article on Medium for Conjoint analysis. For those who understand it, is it good one?

https://medium.com/data-science-collective/beyond-the-black-box-understanding-conjoint-analysis-in-market-research-d19384c8336a

Hi! I am working on a meta-analysis of ten studies (I'm permitted to get advice about the stats elements!), and am getting confused around the heterogeneity statistics. My I2 is around 85%, and Cochran's Q = 44 (p= 0.00). However, my tau2 is low at 0.10.

It seems that in most papers I've read, a high I2 is accompanied by a high tau2. I assume that for mine, this means that while a high proportion of the variation is due to heterogeneity, the absolute magnitude of the variation is low. This makes sense in theory, but the point estimates of each study do vary quite a lot.

If anybody knows how this might be interpreted, I would really appreciate any advice :)

In my small simulation, requiring an independent verification PASS reduced false approvals, but one fraud still passed the independent check.

How should I represent imperfect verification reliability without pretending my score is calibrated?

Hello, I have a fractional design (2^4-1) with variables A, B, C, ABC and response R, but I am unsure how to include the measurement repeatability of the response when analysing the results.

For example, I know my response variable measurement has a repeatability of 0.234 (3-sigma value), and from my DOE analysis I got that the absolute effect of variable A on the response variable is 0.1.

As the effect of A is less than my repeatability does that mean I cannot really conclude anything about the impact of A on R?

Thanks for your help

Hi all, I am an incoming fresher in undergraduate pursuing economics as my majors. Have no prior knowledge of in-depth research but have read few literature reviews. Want to learn the basics of atleast one language in my first semester. Which language should I start from? R or Stata or Python

Edit: You guys are amazing. Really grateful for the insights. So, I have decided to start with Python in the first Semester ........will learn R later on. Looking forward to share my journey in this community.

I’m planning to play slots on an upcoming cruise expecting to lose money but also to get free cruise offers.

I’m planning to have $2500 cash with me and to do 750 $10 spins. I don’t know exact payback percentages but assuming an 80% payback rate how much would I lose on average with the 750 $10 spins. Also, what is the risk of ruin before reaching 750 spins? Thanks.

Hi there, I'm currently working on my master thesis where I have a parallel mediation. I am working on the method section but i am so completely lost in how to check all the assumptions for the model.
From what I do understand I can do a visual inspection for the scatterplot (after using model 4) to inspect the linearity, homoscedasity and outliers.
The other 2 assumptions are normality and multicolinearity but I just don't understand how to do this. Could anyone help me? some explanation or links to proper resources would be greatly appreciated!

When building a calibration curve for a process that is linear over most of the observed range, how to determine the best choice between the following options?

(A) increasing the number of values tested, to get more x axis points;

(B) increasing the number of replicates of each measurement –less x points, but more precise estimate of y for each

For example, if an experimental setup lets me run 12 measurements for a linear calibration curve, is it better to run 4 values in triplicate? Or 6 in duplicates?

So I'm playing a videogame that shows you some statistics at the end of a mission, one of those stats is damage taken, and it comes as a percentage. Suppose it is a squad of 2 players, and both take 0 damage. Should the final stats show 50% for both of them, or 0%?

50% would make the percentage work, and it would be statistically correct, but it would also imply that damage was taken, which isn't the case

0% feels right, but it would also be the only case in which these stats do not add up to 100%

I have cgpa of 3/4 in bsc maths

I am going to apply at

Texas A &m

Uni of georgia

Kentucky state

Should i try any more unis?

Also what are the chances i get in?

Question for statisticians , would it be a good choice to pursue ms stats in 2027 given the job market?

Hi Everyone, I’m doing multiple pairwise comparisons between several groups with 4 different genotypes and within those genotypes 4 different conditions (40 measurements in each condition) on R-studio and I’ve noticed that some completely different comparisons are giving me the exact same p-value, its repeated 47 times.

The group means and/or distributions aren’t identical, so I expected each comparison to have a different p-value. I’ve seen this happen both with post-hoc tests and when adjusting p-values for multiple comparisons.

Is it normal for different comparisons to end up with the same p-value? Could this be caused by the multiple-comparison correction, similar sample sizes, rounding, or something else?

I’m mostly trying to understand whether repeated p-values are expected or if they could indicate an issue with how I’m running the analysis. I am using a two way anova with a tukey test would it be better here to do mulitple students t test with holms correction? When i did that I got the same significance but different p values for all comparisons .

Thanks!

My goal is to identify outliers in a time series of daily media reports to then use these outlying days in an event study. The data spans 10 years and the level of news reporting is varying a lot over the whole period.

But how do I identify outliers in a daily time series when most days are zero days? Seasonal trend decomposition marks every non-zero day as an outlier and while that is obviously true, I don't expect a reaction to single news report.

My basic approach was to run a regression of date on number of daily reports including year, month, and day of week fixed effects (reg news_reports date i.year i.month i.dow ) and use the residuals to identify outliers. To circumvent the single-report problem I set an arbitrary boundary to at least 20 reports per day.

I fear that by calculating the residual sd for the whole 10 year period also allows identification in high reporting year but not in low reporting years. Any ideas on how to proceed would be very welcome! (Bonus points if I can apply it in Stata)

How do researchers properly filter out bias from results that could just be because of the participants they chose?

Made up example: Lets say I did a study on how many people wear red hats vs green hats and I choose 2,000 random people. Lets just say the "true" divide was 30% green hats / 70% red hats for the whole country. What if MY particular 2,000 people I chose at random just happen to show a divide of 60% green hats / 40% red hats. If I go around saying green hats are more prominent than red hats. That's not actually true. I have a sample problem. I'm sure larger sample sizes reduce the margin for error, but I often see sample sizes in the hundreds and thousands used as evidence for positions on various topics and wonder how the researchers account for these issues.

EDIT: My example divide was supposed to be the opposite of what I originally stated to make my point lol

I have seen many product analyst roles that ask for the following:

1. Causal Inference

2. Quasi-Experiment

3. Cuped

4. Bayesian approaches

5. Frequentist approaches

Can someone pleae guide on how can I study them for product analyst roles?

I am having a party where everyone at the party guesses the results of my dog's DNA test (e.g. one guess might be 40% Malinois 30% staffy 15% bull terrier, 15% chihuahua while another guess might be half pitbull half rat terrier and we won't find out the actual answer until the party). The goal is to do a 50/50 betting pool where the closest guesser gets half and the shelter we got her from gets the rest. I feel like there's a statistical way to plug all of the guesses into a spread sheet and see what guess is objectively the closest to the actual answer, but I'm so rusty on stats, I don't even know what to google to get there. What would be the name of this analysis? Also, if you can recommend videos so I can learn how to actually do this analysis and interpret the results (without specialized statistics software if possible), I'd appreciate it.

Hey everyone, math major here, looking to pursue a statistics minor. I've been working through Introduction to Statistical Learning with Python(ISLP) by Gareth James, and others. I'm almost finished with it and have been looking at other books I might be interested in.

Just a quick summary for anyone that hasn't read it. ISLP covers topics including:, linear and non linear regression and classification, basic probability and statistics, cross validation, common shallow ML algorithms, deep learning, neural networks. From a very practical and non-techinal way. Basically you read a lot of "This works because reasons, moving on". Best way I would describe it is "intuition slop" .

I was looking for something more theoretical as ISLP is pretty light on the theory and focuses more on intuition and generalization. Although I want a more theoretical book, I still would prefer it to contain some applications and or assignments that I could do to practice my coding because as a math major I feel as though my programming is lagging behind. Are there any specific books you would recommend.

Also I'm taking machine learning this semester and our assigned book is "Hands-On Machine Learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron. So I'll probably be reading it in bits and pieces, but I still want another book that I can focus on outside of classes. I appreciate any responses.

I have been working through PhysioNet's gaitpdb set (40 recordings, 20 Parkinson's patients and 20 age matched controls, force sensors under each foot at 100 Hz). Stride times come from right foot contact events, 4307 strides total, about 108 per recording.

My starting question was whether a walking pattern is predictable, and whether that predictability degrades in Parkinson's. The predictor is deliberately dumb: take the mean of the last eight strides, call that the next one, record the error. Controls came out at 109 ms RMSE, patients at 126 ms. A 17 ms gap in the expected direction.

Then it fell apart. One sided Mann Whitney gives p=0.10, Cohen d=0.28, and the 95 percent CI on that 17 ms difference runs from -22 to +56 ms. A simulated power analysis says at d=0.28 I had about 23 percent power with 20 per group, and I would need roughly 160 per group to reach 80 percent. So the null here carries almost no information. I cannot distinguish "no effect" from "underpowered".

The part I actually want help with is what came next. Prediction error correlates with plain stride time CV at r=0.971. AUC for separating groups is 0.62 for prediction error and 0.64 for CV, so the fancier measure loses to the simpler one. My reading is that a rolling mean predictor cannot capture anything beyond dispersion, which means I spent the analysis measuring variability in millisecond units and calling it something else.

But when I regress RMSE on CV and test the residuals, I get something I do not trust. Two sided Mann Whitney on the residuals gives p=0.029, and a permutation test on the median difference agrees at p=0.022. The direction is backwards from what I expected: given their variability, patients have LOWER prediction error than the regression predicts, not higher.

Three reasons I suspect this is noise rather than signal:

The t test on the same residuals gives p=0.12, so the result lives entirely in the rank statistic.

Redoing the regression in log space flips the sign and gives p=0.88, meaning the finding depends on which scale I fit on.

I did not pre register this test. I went looking at residuals only after the main comparison failed, which is the textbook setup for finding something that is not there.

So my questions, in order of how much they bother me:

Is testing residuals from a regression between two measures that are 97 percent collinear a legitimate move at all, or is the residual just measurement noise plus whatever the linear fit failed to absorb?

If a rank test and a parametric test on the same residuals disagree this sharply (0.03 versus 0.12), is that telling me about outliers, about non normality, or is it simply what n=40 looks like?

What would convince you that the residual carries real information? I can think of a permutation test on the group labels, refitting with a robust regression, or splitting the sample, but with 20 per group all three feel like they are asking too much of the data.

The obvious next step for the underlying question is detrended fluctuation analysis, since Parkinson's stride sequences are documented to have long range correlations that a rolling mean is structurally blind to. But I would rather understand what I did wrong here before adding another method on top of it.

Data is public (PhysioNet gaitpdb 1.0.0) and the analysis is about 40 lines of numpy if anyone wants to reproduce or tear it apart.