This is the wrost part of the entire quest to be honest. Once you get through this, the rest isnt too bad.
As of September 2022 on the Yelinak server, there are three spawn points for this guy:
1. Static mob on the 4th floor landing under the torch. Timer is 14.5 minutes
2. The pather mob that spawns near the top of the stairs going to 4th floor. Timer is 8.5 minutes (spawn point moves slightly but you should see him)
3. The pather mob on the 3rd floor that spawns on the marked point on Brewalls maps. (others mentioned the loc in previous comments).
About 4 hours of killing, I got all the skulls i needed.