HelpData Analysis with Python

pandas merge keeps duplicating rows

Omar Haddad

Posted Yesterday

I'm merging orders with customers and the result has more rows than the orders table. I'm using `how="left"` on customer_id. What am I missing?

9 replies

Accepted answer

Mei-Lin OtaInstructorYesterday

Your customers table has duplicate customer_id values, so each order matches more than once. Run customers.customer_id.duplicated().sum() first, and pass validate="many_to_one" to merge so pandas warns you next time.

9 replies

Showing 4 of 9 · sorted by oldest

Omar HaddadAuthorYesterday
That was it — 42 duplicate ids from an old import. Thank you!
Lucas Moreau20h ago
Didn't know about validate=, that's going in my notes.
Hana Sato14h ago
Same thing bit me last week. drop_duplicates with keep="last" fixed my case.