Can't connect error 2002

This is a bit of a newbie question --

I've just installed MySQL 4.1.13 on a Fedora core 3 system. It
installed great. It wasn't hard to get it running and create a
few databases. But I can't seem to make the client shell run
from anywhere but the mysql/bin directory. I've placed the
/usr/local/mysql/bin directory in my PATH, but I still have
to change to this directory to make the MySQL shell to work.
If I'm try to run it from any other directory I get this
error:

ERROR 2002 (HY000): Can't connect to local MySQL server
through socket '/var/lib/mysql/mysql.sock' (2)

What sleight of hand to do I need to do the get the MySQL
client to run no matter where on the machine the calling
process is? Since I'll need to do this to get my web server
to use MySQL.

Thanks in advance.

B Squared