Oriskany/Herkimer Co

Williams genealogy - anyone know if there are birth/baptism/marriage type records for Oriskany/Herkimer Co NY? Can't get past my James Williams generation - don't know his wife's last name - he was supposedly a Roger Williams descendant & one of his son's marriage says "no closer in relationship than 1st (crossed out) 2nd cousins! Thank you.