Is digital marketing still relevant or useful even if we don't sell our product/service online?

Yes, digital marketing is still useful as it helps in CRM.
And also one is able to know and understand the needs and requirements of their customers.

Digital marketing (in my opinion) is the next big thing.
For instance marketers will be able to monitor things accurately like what (and how long) an ad is being viewed, how many sales conversions occur, and also what content works/doesn't work.

Without a doubt, while your product/service is not available online, your customer's experience with your brand could begin in the digital world.
Take for instance any product or brand that may require for your customer to do some research on you; the first likely port of call will be a search engine or social media.
If your customers are asking about you online, you need to make sure you are available to them.

It is indeed.
Having your business online is as important as having a Customer Services department.
Your presence on Digital Media is a reflection of your brand while social media presence is your online representative.

This is based on the preconception that websites are primarily for e-commerce, or only useful if a business involves some element of online selling.
When the function of a website is viewed as selling the business as a brand, including brand values and personality; all of which a customer buys into then, an online presence can begin to be seen as important and a necessity.
The majority of consumers 'Google' a product or service before buying online or from the high street, so in order for customers to be aware of the existence of your company, it is necessary to have an online presence.

Generally yes, digital marketing can be useful and can affect your offline operational company. One hot topic to think about is how companies are bridging their online and offline marketing efforts and connecting them to offer a more accurate means of reporting. Your offline sales can be affected by an online action. For example your potential customer was on FB, saw your ad, liked it and decided to check out your page. Got the business phone number or filled out a form and then made a purchase by visiting your physical store. 

Absolutely yes, because if your business is offline, you will be shocked when you know that 67% of your customers are online and take their decisions through google search and online reviews!

Digital marketing can work for any kind of business as long as you understand well your target audience and know where and when to find them.