Identifying the weak link in your system

Our systems are only as good as the weakest link in the chain, so how do you identify which component that is? Further, how do you figure out what to replace it with on a tight budget?