Questions to ask AI providers*
- What is your AI’s core technology and architecture? You want to understand what type of AI model the vendor uses, whether it’s deep learning, reinforcement learning, supervised learning, etc.
- What data does your AI require for training? This question helps you to understand the quality and quantity of data required by the AI model.
- How do you ensure your AI model’s accuracy? It’s important to know how they test and validate their models. Ask them to provide case studies or test results.
- How does your AI handle bias and fairness? AI can inadvertently perpetuate bias if not carefully managed. Ask how they mitigate these risks.
- How is your AI model updated and improved over time? Models need regular updates to stay effective. Ask about the process for this.
- What is your AI’s interpretability and transparency like? Some AI models, especially deep learning models, are often described as “black boxes” because their workings are not easily understood. Ask how they ensure transparency in their AI processes.
- What is your model’s performance in real-time applications? Ask about the speed of the AI model, how it handles large data volumes, and how it performs in real-time applications.
- Can your AI model be customized to our specific needs? Depending on your needs, you might want an AI that can be tailored to your unique requirements.
- What kind of support and training do you provide? Find out what kind of support they offer after the AI is deployed. Do they provide training for your team?
- How do you ensure data security and privacy? This is an especially critical question in industries dealing with sensitive information.
- How do you ensure your AI model maintains the confidentiality of our data? It’s important to understand what protections are in place, and how your firm’s data will be handled.
- What is the total cost of ownership of your AI system? Beyond the initial cost, ask about ongoing costs like support, updates, or subscription fees.
- Can you provide references from other clients in our industry? References can provide valuable feedback on the vendor’s performance and reliability.
- How does your AI integrate with our existing systems? An AI solution needs to work well with your current tech stack.
- What is your roadmap for future development? You want to ensure that the vendor’s vision aligns with your company’s future needs.
- Can we conduct a pilot or proof-of-concept project? This will allow you to test the system in your environment before making a significant investment.
- Which cloud provider was selected when developing the AI solution and why? This could provide deep insight into how the AI is grounded as well as overall product development process. (hat tip to CJ Webster for suggesting this question).
- Can you ensure that my data won't be used inadvertently to answer questions from other firms using your service, and if so, how do you ensure this? This is important since you are ethically obligated to ensure that you and your technology providers have taken reasonable steps to ensure that confidential information is protected form disclosure. (hat tip to Richard D. for this suggestion).
* Credit for majority of AI questions: Stephen Backholm, Digital Transformation Expert (LinkedIn)
Questions to ask cloud providers
- How long has the company been around? Is it well-funded or has it been acquired by an established company?
- What type of facility will host your law firm's data?
- Who else has access to the cloud facility, the servers, and the data? What mechanisms are in place to ensure that only authorized personnel will be able to access your data?
- How does the vendor screen its employees? If the vendor doesn’t own the data center, how does the data center screen its employees?
- Is the data accessible by the vendor’s employees limited to only those situations where you request assistance?
- If there are integrations with the company's product, how does the company screen the security processes of the other vendors and of the product that integrates with the software?
- If there is a problem with a product that integrates with the vendor's software, which company will be responsible for addressing the issue?
- Does the contract with the vendor address confidentiality?
- How often are backups performed? Is data backed up to more than one server?
- Where are the servers located? Will all of your firm's data always stay within the boundaries of the United States?
- What type of security is used at the data centers where the servers are located?
- What types of encryption methods are used? Is your data encrypted while in transit and while at rest?
- Are there redundant power supplies for the servers where your data is stored?
- If a natural disaster strikes one geographic region, would all data be lost or are there geo-redundant backups?
- If there is a data breach, will you be notified? How are costs for remedying the breach allocated?
- Does the contract include a guarantee regarding uptime?
- What remedies does the contract provide?
- Does the agreement with the provider contain a forum selection clause or a mandatory arbitration clause?
- What rights do you have upon termination of the contract?
- How do you retrieve a copy of your law firm's data and in what format will it be provided?
- What rights do you have in the event of a billing or similar dispute with the vendor?