I can't refer to myself as a perfect English speaker but I believe learning English isn't so hard at all. I think it all depends on your attitude to learning, if you believe it's hard then it's hard. And if you believe otherwise then it's not going to be hard. Once you're clear with the 'your mindset' phase, the real form of learning would come easily.
I'm from a country where English is our general language although not my mother tongue. They start teaching us English right from our nursery school days but this fact doesn't even make everyone better in the language because I could point to a lot of people that went through this process and are still not good in the language. But, I think the problem is all about expression; If someone has this kind of believe you have with the fear of being wrong then learning English won't be easy. Hence, it's advisable to always use forums such as this, and also try to speak the language with people irrespective of you speaking it good or bad. As times goes on you'd get better like they say, "practice makes perfect."
There was this guy I met in high school when I was a tenth grader, he couldn't speak English correctly. When he talked, we'd laugh behind his back by the time he was gone. He couldn't keep mute like others of his kind because he had ideas and he felt he had to express these ideas. He couldn't opt for the local dialect because doing so was a taboo that came with severe punishments. Others with his problem would rather shut their mouth up but he went on expressing himself. To cut the long story short, by the time we became 12th graders he was way better than us(who laughed behind his back).