Sweeney put the visitors ahead early on before Ian Watt shot home from 20 yards after receiving a Cammy Lawson pass to equalise.
It was 2-1 Lanark at half-time thanks to Eliot netting from a Watt pass.
Doherty levelled for ’Hill early in the second half but Lawson shot the hosts ahead again on 70 minutes.
A Sweeney tap in made it 3-3 before home skipper Lawson was sent off for a second booking.
’Hill won it on 88 minutes when Sweeney netted again to break Lanark hearts.
Second bottom Lanark, on six points from 11 games, visit Yoker Athletic in the league this Saturday, KO 2pm.