Yea, they used to use a chalk paste on them, Soak it for a hour or so, scrub it with a stiff small brush, a fingernail brush or a nylon paint stripping brush.
What does the water look like? And what cleaning solution are you using?
Keep changing the water and soap until it says clean after soaking and scrubbing.
I like to use a combination of Biz and dollar store Cold water wash, (Woolite), 1/4 cup for 1.5- 2 inches of water in a Cement mixing tub, probably a gallon of water. Rinse well between water changes. Too much soap will cause more problems.