One source of confusion for students when they reach college and begin to do college-level mathematics is this: in high school, it is usually pretty apparent what formula or technique needs to be applied, as much of the material in high school is computational or procedural. In college, however, mathematics becomes more conceptual, and it is much harder to know what to do when you first start a problem. As a consequence of this, many students give up on a problem too early.
If you don't immediately know how to attack a problem, this doesn't mean you are stupid,
or that you don't understand what's going on; that's just how real problems work. After all, if you already know how to do it, it's not really a problem, is it? You should expect to be confused at first. There's no way you can know ahead of time how to solve every problem that you will face in life. You're only hope, and therefore your goal as a student, is to get experience with working through hard problems on your own. That way, you will continue to be able to do so once you leave college.
If you already know how to do it, it's not really a problem.
One of the first steps in this is to realize that not knowing how, and the frustration that accompanies that, is part of the process. Then you have to start to figure out the questions that you can ask to help you to break down the problem, so that you can figure out how it really works. What's really important in it? What is the central concept? What roles do the definitions play? How is this related to other things I know?
Sometimes, working through specific examples can be a help. If a problem involves variables, or named constants (like a, b, etc.), then give those variables some specific values and see how things work in that case. Try to figure out what "special" values for these variables make the solution easier (or harder); that can tell you some valuable information about the problem. Once you can handle a specific case, try to use that understanding to work on the more general solution.
Don't make the mistake that so many do, which is to look at the answer in the back of the book as soon as it gets hard. This cheats you of the opportunity to solve the problem yourself (which is when true learning occurs), and trains you to solve problems in a way that you will not be able to maintain in later life. If your only method of working a problem is to know the answer ahead of time, how will you be able to perform on an exam? Or in real life, where there is no back of the book? Your only skill will be a useless one, so don't train yourself to rely on it.
The important thing is not to give up. If you need to, stop for a while and work on something else, then come back to the problem again. Often, when you start again, you will see something you didn't see before.