Without more facts, it is impossible to determine with certainty whether we end up keeping those items by design, or whether it is simply due to a missing script entry. Convincing arguments could be made either way.
With most quests, extraneous items automatically disappear from our inventory, once the related quests have been completed. But not always. Lily's quest would be one example in which the quest items remain after completion. The sewer key card from Act 3 would be another such example. And there are a few others.
Since the main goal of the True Patch is to fix LEGITIMATE bugs, and since there is no clear indication that these "leftover items" constitute a legitimate bug, I was forced to abide by our philosophy of "if it ain't broke, then don't fix it." Hence, the True Patch makes no changes to the related scripts in these cases. There is no compelling reason to do so.
Conclusion: as I have said many times around here, if the player desires what must be referred to as arbitrary changes in Bloodlines, then such changes belong in a mod -- not in a patch.
:police:
As for me personally, I have always considered those leftover items to be "keepsakes," as Schu has suggested. I just toss them into our desk drawer in Santa Monica, or into our foot locker in the Downtown haven, or stash them in the refrigerator... etc.