Expert elicitation is one of the tools used to support and assist conservation decision making. There are several features of expert elicitation which make it useful; as a form of ‘collective intelligence’, it uses available resources efficiently, and it can reduce bias and improve accuracy of quantitative estimates. In this project, we used expert elicitation to collate information on the benefits and costs of management activities for biodiversity offsetting. In our first case study, our results were inconsistent with some of the available empirical evidence on efficacy of management actions for the malleefowl. Here, we describe the lessons we learned, and provide recommendations that could help minimise bias when expert elicitation is being used to help assist conservation decision making.