So, first of all, what exactly is triclosan? Triclosan is an antibacterial and antifungal agent that inhibits fatty acid synthesis in many bacterial species. When triclosan hits a bacteria, it will bind to an enzyme called enoyl-acyl carrier protein reductase, or ENR. When ENR is bound to triclosan, its affinity for another coenyzme, nicotinamide adenine dinucleotide (NAD+), is greatly increased. This means that they become best buds and bind a bunch (yay alliteration!). As a result, instead of helping to make lovely fatty acids for the bacteria in question, everything comes to a grinding halt in a big ENR-NAD+-triclosan complex that just hangs out instead of doing useful shit. Since the bacteria can't synthesize fatty acids, it can't built its cell membranes, so it dies. Because humans don't use ENR to make fatty acids, it is safe for us to use.
Environmental activists have raised concerns about triclosan's potential polluting effects. These concerns mostly center around aquatic life. For example, triclosan seriously fucks up algal communities. There is also evidence that triclosan bioaccumulates in fish, meaning it is absorbed at a faster rate than it is excreted. The Environmental Protection Agency (EPA) has also found waterways where triclosan levels reached concerning amounts for aquatic plants (although not for animals). In sufficient quantities, triclosan can be toxic to aquatic life.
No large scale ecological studies have sufficiently investigated the potential harmful environmental effects of triclosan, but there is definitely evidence suggesting that it may be a concern. The EPA suggests that industrial use of triclosan, as opposed to commercial use, is the biggest concern ecologically, and they center their suggestions for future research and their recommendations based on that assumption. In comparison to triclosan pollution of that magnitude, your face wash doesn't seem particularly sinister.
However, as they say, no single drop of water thinks that it is responsible for the flood. You probably don't need triclosan in your life, so it may be worth it to reduce or eliminate that particular ingredient from your routine. Still, if you are concerned about environmental impacts of triclosan (or anything else!) you'll likely have a greater impact by contacting your congresspeople (The Directory of Representatives can be found here) and letting them know about your passion for limiting industrial pollution and by voting for environmentally friendly measures when they make their way onto your ballot.