We were very tempted to buy one, but I knew we could not handle doing the maintenance. There are so many ways a rental property can bite you. The tenants can not pay the rent. The property can be empty for a few months. You can have unexpected repairs. The tenants can trash the place. And you can’t put off repairs to a rental the way you can choose to do with your own house. I lived with an un-usable second bathroom for quite a few years, for example, while I saved up cash, but it would have been really hard to rent an apartment that way. And I’ve seen tenants ignore leaks and otherwise mess up a place.
A friend of mine owned an apartment in a bad neighborhood (rule of thumb: NEVER buy a building you wouldn’t want to live in yourself) and he ended up with a tenant who was a drug dealer. She had a small child and he was not able to evict her immediately even after she stopped paying rent. He had to pay his lawyer while hers was covered by some low-income program. The crowning insult came when the police came to the apartment and broke the door down. My friend had to pay for the repairs and he *still* couldn’t evict her .It took him forever to get out of that building (a less ethical person probably would have had it burned)
Of course, most tenants aren’t like that, and in a duplex at least you live right there and can tell what’s going on, but you never know. It just introduces a lot of risk and unpredictability into your life. I don’t think I’d want to go investing cash in real estate until I had a really sizable cushion to play with.