So I talked to my psychiatrist about my memory problem, this is what he told me:
The reason is anxiety, it´s part of your illness. You could better it by diminishing your anxiety. Go for walks, do relaxation techniques, avoid too much caffeine, don´t drink more than two cups of coffee a day-that can sharpen you but more than that increases your anxiety and makes your concentration and memory worse. This is quite common with mentally ill people. Although you don´t feel the anxiety it´s still there. Gingko beloba is just like having two cups of coffee.
And then about my fear of being attacked or ridiculed in public by strangers: Just run away, don´t let that stop you from going out, that´s going way too far. It isn´t any more likely that it will happen to you than anybody else.