So on, say, an 8GB system, you really want to restrict SQL to only using say 6GB. Then we have plenty of headroom for the SQL console, ApexSQL log, backups and any other software you need.
To do this, execute the following SQL commands:
EXEC sp_configure 'show advanced options', 1
RECONFIGURE WITH OVERRIDE
EXEC sp_configure 'max server memory (MB)', 6000
RECONFIGURE WITH OVERRIDE