I have created a test team site with a document library name "My doc lib" and a test user who does not have access to the team site, as shown below
Here are the steps to configure the Anonymous access to the Lists and Libraries(ex., My doc Lib) and before proceding to the steps, lets login with the new user and try to access the document library
Ok, test Passed :) , Document Library Access is denied for the test user and here are the steps.
Step 1: Go to the library settings of the document libraryStep 2: Click Permission for this document library
Step 3: Now the permission Tools page will be displayed, but this page(Ribbon) will not display the anonymous access option, because this libary is inheriting the permission from its parent, and in this scenario it inherits permission from the Team site.
Step 4 -Stop Inheriting Permissions from the parent
Step 5: Now the anonymous access option is enabled in the ribbon
Step 6: Click on the Anonymous access button on the Ribbon, and click on View item.
Note: Anonymous users cannot Add, Edit, Delete items in the document library, they can have only view permission.
Step 7: Login with the test user to verify whether the anonymous access is enabled
Done!, now the anonymous user is able to access the document library, see top right corner.