The simplest method to do this that is oopsproof is to use a keyed jumper plug (that only fits one way) that connects the batteries in series to the controller for riding. When it is removed, neither battery is connected to anything (including it's counterpart).
Then the charger plug uses a connector identical to the jumper plug, but is wired to connect the batteries in parallel to the charger.
A side benefit is that if you needed an antitheft device, the jumper plug, when removed, would prevent the electrical system from operating.
I recommend using a connector that is relatively heavy duty and does not unplug easily, simply so you "can't" have connection issues with it while riding.
Personally, I'd use Anderson SB50 (or larger if you need really high currents; I use them for over 100A just fine), with teh housings bolted together top to bottom to make a keyed block, with a T-handle between them to pull it out with (it'll take a fair bit of force, so you want to bolt the battery-end-connector to the bike frame or heavy duty battery box wall).
You can also use the Anderson PP75s, they are the same contacts but in modular housings that can be setup to be keyed in different ways.
The biggest advantage to the Andersons is they have sacrificial tips so if you get a spark while connecting things up, it doesn't affect the actual contact surface. All of the ones I have are used off of other things (powerchairs, UPS equipment, etc), and still work fine (I reuse the contacts by keepng the existing crimped-in 8-10g wire in there, and connecting to that, usually).
The disadvantages to these (besides cost) is they are rather large for the current capability, require you leave enough wire out the back of the housing to let the contact float in the housing (or they won't mate correctly and will cause high resistance, with heating problems), and for best contact should be crimped to the wire; you may need special crimpers to do it if you don't have something of appropriate size already.
YOu could use bullet connectors of various types, but you'll have to work out a method for keying the connector so it cannot be plugged in wrong.