Grinding any remaining stumps is always recommended. Not only will it improve the aesthetics and usable area of your yard, it will protect your lawn equipment, visitors and children from damage or injuries.
Leaving a stump to decay will attract beetles, termites, ants and other wood-boring pests. You may not mind them in your yard, but they can eventually spread to your home.
In addition, residual stumps will often sprout new growth taking nutrients from the plants you actually desire.