It could be the type and brand of soap you are using. I have notices some soaps will remain in the brush longer than others. Also are you using any lather enhancers like glycerin or oils?
An occasional warm water and Vinegar soak and rinse, 30 min soak then shampoo with a good hair shampoo and a dab of conditioner works wonders for a brush.
Lately I have been combing my brushes with great results. Start at the tips to untangle the hairs then work your way down and around the brush. Comb when dry and do it gently so as to untangle and not pull out hairs. Just takes a few seconds
When combing I noticed some brushes have more dried soap than others. I suspect it is the soap brand and type, as I recently purchased several new soaps and have been sampling them. A vinegar soak and rinse, shampooing and combing make a big difference in brushes looks and feel. They also seem to lather better.
Hard water may also be an issue.