I am working on a diabetes prediction model for my project and I need help on how should I handle missing values in the smoking history column in my structured tabular dataset. <\/p>\n
My dataset has 100,000 rows, with around 35% of rows having “No Info” for smoking history. Since smoking history has a significant impact on diabetes, this column cannot be ignored. <\/p>\n
Other entries in this column are: “Never”, “Current”, “Not current” and “Former”<\/p>\n
Key concerns:<\/p>\n
Encoding: If I am encoding this column, then how should “No Info” be treated in this case? One hot encoding will lead to unneccessary high dimensionality whereas there is no clear order that I can choose between the values if I go with ordinal encoding.<\/p>\n
Data Loss: Would dropping these rows (35%) lead to bias, or is it a valid approach?<\/p>\n
I would appreciate your personal insights on the best approach for this since I have already searched this thing enough on the internet.<\/p>\n<\/div>\n
submitted by \/u\/shaitaanbaluck <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
