Labeling and documenting data is the key...
Plants of the same genus look similar enough to one another that you may be able to make a good guess, but seeds vary considerably. Some seeds that are not even remotely connected look alike.
When labeling plant tags, always use a pencil or a botanical pen or marker.
Permanent markers tend to fade over time. Check your tags regularly as you walk through your garden.