All of the ones I have put on bikes are smooth but can produce violent torque at a dead stop in 20" tires. So you have now verified all wiring to the motor? You can use the 6 fet to test under load, just keep the voltage to 50v or lower and program the controller for 30a batt 70a phase max. I have done this and the controller worked fine. I tested as high as 60a batt 80a phase, but I don't suggest that. I still have that controller and it can run sensored or sensor less. I can try it out sensor less and let you know how it rides. Well I might not be able to yet as I am stress testing controllers using an 18.6 KV Cromotor.
Your video of it under load sounds like its on 2 phases from bad phase / hall connection. I have done this a few times and that is what mine sounded like. Every sing time i though i had a solid connection on both but closer inpection eventually revealed wiring issues. Either that or you have the worlds loosest spokes. Does the motor have any physical wobble to it on spin up with no load? 1 or 2 km is normal side to side.