Most of the ANBU serve mainly as the FBI, the CIA, and sometimes the police force of a village. They work covertly and secretly. Things of that matter. Most of them are also just elite. Somewhere between a regular Jonin and a Tokubetsu Jonin (examples: Ibiki, Anko, Hayate). Maybe even higher. The ANBU are all pretty strong. Though the series have abused the ANBU as mere fodder at times, but you have to remember who exactly kicked their asses: Kabuto, Orochimaru, Gaara, and people around that range of power. I completely ignore filler arcs so any butt-whoopings that are directed towards the ANBU don't count.
Anyway... the ANBU are the elite. Yamato, Sai, Kakashi, Itachi, and plenty of others have shown how capable they are.