yes, you should replace those low voltage cells, the first step to do a battery pack is cell sorting, to make sure cell consistency.
voltage difference should be less than 5mv.
because faulty cell lowers overall voltage, causing early cut-off
Weakest cell is stressed most; stack deteriorates...