I believe that you are talking about labelling all of TeamProject and not just creating a label at TeamProject scope ($/<TeamProjectName).
In any case, you should be able to see that label when you click Find button in "Manage Labels or "Find Label" (new name for the dialog brought up by File > Source Control > Find Label... menu) dialog.
What build you are using? Please make sure that Project dropdown is set correctly.
Thanks,
Bhavik
I think the issue has to do with being able to apply a label on a folder that doesn't have any files in it. While it really doesn't make sense to use labels in this way, it might be good to warn the user that the label will not be applied.
I just created a label using UI on Empty Team project folder and it worked fine. I could see the label in UI and at comamnd line.
Thanks,
Bhavik