There are a couple of misconceptions about self confidence and self worth that can be really confusing.
First, you can't build self worth and you don't have to. You are already worthy. It is the basis of who you are. Look at animals. Every animal knows it is worthy and that their needs matter. It's simply who they are.
Only humans sometimes have thoughts that doubt there self worth. But that doesn't make you any less worthy. Your self worth is absolute and you can't decrease it. You can only be misled by negative thoughts. But those are like dark clouds moving in front of the sun. The sun is still shining and so is your worthiness.
Second, self confidence is about acting from a realization of self worth. It doesn't mean you have to be good at something. You can even have self confidence while failing. Especially if you look at your failures as 'unscheduled learning opportunities'.
Most importantly, SELF confidence has nothing to do with other peoples compliments or criticism.
Sure, compliments are nice but how people judge you sometimes has very little to do with you. So I take the compliments and criticism, but my main focus is on how I feel about my efforts, not my results. Because effort I can control, results are dependent on many factors.
This is what works for me. Hope it resonates with you. :)