Below is a video tutorial for creating and using FB Custom audiences.

Note: If Audience Size is too Low, You cannot remove users from that audience because it will result in a low audience size and may result in under-delivery or non-delivery of your ads. This is a restriction put in place by Facebook.