We did this on saturday, all we did was kill everything in the fort and got a druid to track and pull anything wandering around the place. We got 2 pops of the patriarch and both times it dropped the monocle. This was in about 45 min so i think it is just a totaly random drop