1. What does New_Uri() method actually return to MapBasic? Is it a URI object or a .NET references to an image? 
For instance:
Call SetRbnBtnCtrlSmallIcon(BA_Btn, New_Uri("pack://addinorigin:,,,/d:\Core\Scripting\Libraries\btnImageLib.dll;component/Images/Tools/Options/OptionOneSm.png", 0))

This is .net URI class constructor declaration in MapBasic that basically invokes the constructor of URI class. 

2. Also how does it actually work? Does it get each image from the dll and load it into memory, then somehow make a reference to it?

This loads the dll from URI and then loads the specified image from the dll into memory as byte stream.

3. I haven't gone right through the MapBasic V16.0.1 documentation, but is New_Uri() primarily used to get images only?

New_Uri() is just invoking the constructor of Uri class and creating object of it. 
It can be used in normal .net functions also where Uri is needed.

4. When forming the URI with a local path like the following it works fine, it forms an absolute path to the dll
    Call SetRbnBtnCtrlSmallIcon(BA_Btn, New_Uri("pack://addinorigin:,,,/C:\inetpub\wwwroot\biz_e_asset\AMS_Final\ams_final_scripts\Libraries\BAResources.dll;component/Images/ButtonImages/Tools/Trace/TraceSetDistanceLg.png", 0))

However, when using a server path instead of a drive letter path like the following:
    Call SetRbnBtnCtrlSmallIcon(BA_Btn, New_Uri("pack://addinorigin:,,,/\\CSSVR3\AMS_Final$\ams_final_scripts\Libraries\BAResources.dll;component/Images/ButtonImages/Tools/Trace/TraceSetDistanceLg.png", 0))
   It throws an error and looks like the New_Uri() method interprets this as a relative path.

To use network path you have to provide a path like this and the user who is running the application needs to have modify permission on UNC path.
Call SetRbnBtnCtrlSmallIcon(BA_Btn, New_Uri("pack://addinorigin:,,,/file://CSSVR3/AMS_Final/ams_final_scripts/Libraries/BAResources.dll;component/Images/ButtonImages/Tools/Trace/TraceSetDistanceLg.png", 0))

addinorigin is our custom authority for pack URI and loading images from dll.

Also, loading an assembly from a path with space will fail and this has been identified as a bug since we are not un-escaping a URI string while loading assembly. 

5. In MapInfo Pro 64-bit versions 12 to 16.0.0 we cannot use this functionality as it does not exist.
If I create a method in one of my libraries, in the above MapInfo Pro version, that returns a URI object with MapBasic syntax, could this work?

The support for loading images from dll was added from 16.0.1 version of Pro.
In older MapInfo Pro versions, you cannot load images from dll on ribbon even if you provide the MapBasic syntax.