I'm leaning toward fungal. Are the plants in a very moist environment?
Exactly what I was thinking altho, the brown wouldnt be typical in my book more a black and usually you either have a very mean fungus or you need some sort of injury on the plant to aow the fungus to get a hold on. My money is on Cercospora sp.
positive identification and therefore ruling out of Ramularia and bacterial attack is done by inspecting older infected areas with a magnifying glass, if you find small black dots in the affected area you can be certain you have Cercospora. Also typical for all fungi would be any kind of growth (white / grey / black fuzzy) on the underside of the affected spots. If you have a fungus or any kind of organism make sure you understand how it spreads around before you act as you dont want to spread it unecessary.
However: I wouldnt be afraid to remove leaves on theese, better too many than too little, then bag the pot in, take it in the sink or shower and wash and scrub it down, better too hard than to gentle. Plants are suprisingly forgiving when it comes to physical treatment (shouldnt snap or scratch it open tho) and pests are suprisingly affected by just removing whats home to it and also washing the plant to remove any eggs spores and the likes.
Dont be afraid to use soap or mild bleach or H2O2 on your plant as long as you wash it well afterwards and bag in the pot really well so nothing gets into the pot. Yet if something does, you can flush it.