The small amount of ADA (~1.5ADA) that is charged to send Native Tokens (WMT) is a fee that is set by Cardano/IOHK. At this moment in time, there is no way to get around paying this fee when transacting with Cardano Native Tokens.
Cardano will be releasing Babel fees in the future (https://iohk.io/en/blog/posts/2021/02/25/babel-fees/) which means fees will be charged in the native token, which should make things easier when transacting in Cardano Native Tokens.