people who visited the site but did not opt in > you can use negative audiences for your ads with facebook ads, you just take visitors to the site and substract the people that visited the thank you page. For people who did not purchase, you simply can't with affiliate marketing unless the vendor adds a conversion pixel of yours on their thank you page.
For a slightly different take on the subject and figuring out how to get targeted traffic onto your promoted links, you must have a look at Affiliate Marketing: Instant Traffic to Affiliate Links. A quick look at the reviews will help you decide if this is the exact course you are looking for. Sometimes the right approach is what is required to make a campaign work, this one might just work for you.
Affiliate marketing is also called "performance marketing", in reference to how sales employees are typically being compensated. Such employees are typically paid a commission for each sale they close, and sometimes are paid performance incentives for exceeding objectives. Affiliates are not employed by the advertiser whose products or services they promote, but the compensation models applied to affiliate marketing are very similar to the ones used for people in the advertisers' internal sales department.