I've been a tester for a program called Dota Secure Server, DSS
for short.
DSS can only work, when the other players agrees to install DSS and
play with it on as well. DSS works by scanning the programs that
are activated and make sure no hacks are activated and thus
providing you a green access to the game. Players with hacks on,
will give a red icon to the host and you'll not be allowed to play
in game. If you decide to on the hack in mid-game, DSS can detect
it and the host can kick you out of the game.
But too bad, the program is slightly buggy as it was first stage,
can be easily bypassed as long as you know what you're doing. (But
I doubt people will go that far out to bypass just to play in a pub
game, in house maybe.) and no one uses it.