That's a good general rule. But if someone is involved in an incident in which he used force against another person, and believe that he acted in justified self defense, keeping completely silent may not be the best idea.
But Don't Say Too Much.
Call 911. Be the first to report the incident and do so immediately. If you don't report it, or if there's a long delay, you will appear to have a guilty conscience.
Then, having taken LFI-I with Massad Ayoob, spending time with him and helping with a class of his in Sierra Vista, AZ not too long ago, I'll go along with his recommendation for when the police arrive.
 While one has a right to remain silent, clamming up is what the bad guys do. Following a self defense incident, you'll want to act like one of the good guys. You also won't want the investigating officers to miss any evidence or possible witnesses. What if the responding officers miss your assailant's knife that you saw fall down the storm drain? What if they don't know about the guy you saw pick up your assailant's gun and walk off with it?
 At the same time, you don't want to say too much. You will most likely be rattled. You will also most likely be suffering from various well known stress induced distortions of perception.
 So Massad Ayoob recommends:
* Saying something like, "That person (or those people) attacked me." You are thus immediately identifying yourself as the victim. It also helps get the investigation off on the right track.
* Saying something like, "I will sign a complaint." You are thus immediately identifying the other guys(s) as the criminal(s).
* Pointing out possible evidence, especially evidence that may not be immediate apparent. You don't want any such evidence to be missed.
* Pointing out possible witnesses before they vanish.
* Then saying something like, "I'm not going to say anything more right now. You'll have my full cooperation in 24 hours, after I've talked with my lawyer."
Pleading Self Defense is Very Different From the Common Lines of Defense to a Criminal Charge.
A lot of folks point to the "Don't Talk to the Police" video that is making the rounds on gun boards. But it is about a police contact in general. It works fine when you aren't claiming self defense, and it's up to the State to prove your guilty beyond a reasonable doubt. But things work differently if you are pleading self defense.
 The prosecutor must prove the elements of the underlying crime beyond a reasonable doubt -- basically that you intentionally shot the guy. But if you are pleading self defense, you will have admitted that, so we go to step 2.
 Now you must present evidence from which the trier of fact could infer that your conduct met the applicable legal standard justifying the use of lethal force in self defense. Depending on the State, you may not have to prove it, i. e., you may not have to convince the jury. But you will have to at least present a prima facie case, i. e., sufficient evidence which, if true, establishes that you have satisfied all legal elements necessary to justify your conduct.
 Now it's the prosecutor's burden to attack your claim and convince the jury beyond a reasonable doubt that you did not act in justified self defense.
Let's go through that again.
In an ordinary criminal prosecution, the defendant doesn't have to say anything. He doesn't have to present any evidence. The entire burden falls on the prosecution. The prosecution has to prove all the elements of the crime beyond a reasonable doubt.
If the crime you're charged with is, for example, manslaughter, the prosecution must prove that you were there, you fired the gun, you intended to fire the gun (or were reckless), and the guy you shot died. In the typical manslaughter prosecution, the defendant might by way of his defense try to plant a seed that you weren't there (alibi defense), or that someone else might have fired the gun, or that it was an accident. In each case the defendant doesn't have to actually prove his defense. He merely has to create a reasonable doubt in the minds of the jurors.
So in such cases, it probably doesn't pay for you to say anything to the police, at least early on. Let them do the work of trying to amass evidence to prove the case against you. There's no reason for you to help.
But if you are going to be claiming self defense, you will wind up admitting all the elements of what would, absent legal justification, constitute a crime. You will necessarily admit that you were there, that you fired the gun, and that you intended to shoot the decedent. Your defense is that your use of lethal force in self defense satisfied the applicable legal standard and that, therefore, it was justified.
So now you would have to affirmatively present evidence from which the trier of fact could infer that your conduct met the applicable legal standard justifying the use of lethal force in self defense. In some jurisdictions, you may not have to prove it, i. e., you don't have to convince the jury. But you will at least have to present a prima facie case, i. e., sufficient evidence which, if true, establishes that you have satisfied all legal elements necessary to justify your conduct.
Then it will be the prosecutor's burden to attack your claim and convince the jury (in some jurisdictions, he will have to convince the jury beyond a reasonable doubt) that you did not act in justified self defense. And even if you didn't have to prove self defense (only present a prima facie case), the more convincing your story, and your evidence, is, the harder it will be for the prosecutor to meet his rebuttal burden.