Lover pulled away

I found a guy and thought we're gonna be friends, we started to sleep together regularly so I thought we're gonna be something like friends with benefits but I fell in love with him and told him that and he told me he doesn't even want a friendship, he's just a lover and started to avoid me after my confession. We started to talk more again last month but he as started to pull away again. I'm just afraid I scared him away with my big love for him and he started to dislike me. I want to move on because I know he's kind of bad for me (he's very toxic) but the part of me wants to stay. I wanna keep him at least as a friend or acquaintance but don't know what to do.