Report: Peer Review of STEMCELL Technologies Points to Minor Revisions.
How do you view your vendors? Do you think that they have high quality products promoted with honesty and competent scientific support? Or are they providing unremarkable products embellished with marketing flare? There's a reason most marketing materials end up in recycle bins: empty or partially-accurate claims aimed at advancing a company's bottom-line can undermine consumer trust and hinder scientific progress.
At STEMCELL Technologies, we strive for a more honest and transparent relationship with the life science community. This starts with holding ourselves accountable to higher quality standards in product development, communication and support, with the ultimate goal of advancing science overall.
We recently took a radical step towards transparency by asking three independent scientists to review us against our brand motto of “Scientists Helping Scientists". We invited the peer review team to tour our facilities, interact with our staff and question us on our vision, culture and quality standards. Their verdict: Accept, with minor revisions. Read the report below to find out how the peer review exercise was performed, what the review team found and how we are addressing their recommendations for improvement.
The general perception of marketing in the life science industry is that it is inundated by buzzwords promising product quality, ease of use and experimental reproducibility. Through interviews with 15 life science researchers, we found that scientists value these attributes but often feel awash in marketing “fluff" that makes it difficult to separate hype from reality. Scientists look for honesty, humility and a genuine commitment to science in the brands that support them. They seek transparency and shared goals with their suppliers and want to partner with companies demonstrating integrity rather than those just claiming to have it.
To demonstrate transparency in our communications and start a broader dialogue with the life science community, we took the unusual step of putting STEMCELL Technologies up for peer review by an independent team of scientists. We wanted to know whether, by external standards, we are achieving our mission of being Scientists Helping Scientists. After all, integrity in science depends on peer review and, well, we're scientists, too.
After using numerous advertising channels to put out a call for peer reviewers, we received 312 applications from scientists around the world who expressed interest in evaluating our company against its motto of “Scientists Helping Scientists." The applicants shared with us their motivation for pursuing a career in science, the scope of their research and their rationale for participating in this initiative. A committee of internal reviewers evaluated applicants based on their alignment with the vision of the project and English language competence. This included reviewing written applications and video-conference interviews with shortlisted applicants. To remove the element of bias we excluded our collaborators and any applicant who expressed interest in becoming employed by STEMCELL from the candidate pool. We also attempted to give voice to an international and gender-balanced audience.
The final peer review panel was comprised of:
- Dr. Amy Stone, Senior Fellow, University of Washington, USA
- Dr. Craig Ayre, Postdoctoral Fellow, Atlantic Cancer Research Institute, Canada
- Dr. Fiona Frame, Postdoctoral Research Associate, University of York, UK
The participants were flown to STEMCELL's Headquarter facilities in Vancouver, Canada in April 2018. Over the course of three days, the panel toured our facilities and reviewed our data, culture and processes. They met with over 30 STEMCELL staff across numerous departments to broadly discuss the following aspects of the organization:
- Product Development and Strategic Marketing
- Process Development and Manufacturing
- Quality Management System (ISO and GMP)
- Product and Scientific Support
- Scientific Communications and Sales
The peer reviewers also spoke with STEMCELL's founder and CEO, Dr. Allen Eaves, to discuss his vision for the company, elements of its history and future directions.
The interactions between peer reviewers and STEMCELL staff were video-recorded over the course of their entire visit, with some exceptions during meal times and a group biking activity. Additionally, reviewers' comments were captured on video at the end of each day as well as during the Stem Cell Podcast Interview that marked the end of the peer review exercise.
Documentary and Podcast
Following the visit by the peer reviewers, the video-recordings were edited in-house into a short documentary film. The podcast interview was conducted with the hosts of the Stem Cell Podcast, Drs. Kiki Sanford and Daylon James. Stem Cell Podcast is owned and produced by STEMCELL Technologies.
Results and Discussion
Summary of the Reviewers’ Observations
Out of 312 applicants, 3 postdoctoral fellows—Drs. Amy Stone, Craig Ayre and Fiona Frame—were selected to represent the scientific community in a peer review of STEMCELL Technologies conducted at our Vancouver headquarters in April 2018. The peer review team visited our facilities, met with staff across various departments and observed our processes. Some of their observations are summarized in Table 1 below and can been seen or heard in the documentary film or podcast.
Table 1. Summary of Peer Reviewer Observations
“Throughout this peer review process, I expect STEMCELL to show me everything, warts and all. The good, the bad and the ugly. I want to know everything that’s going on and I expect to be able to come in and examine STEMCELL and ask those questions.” —Dr. Stone, documentary
“Even bringing us in, you have to admit, that it’s a risk to open yourself up and say, ‘I want you to open every door, I want you to shake every cupboard.’ We’ve spoken to more people than I can remember. We’ve been in every department. We’ve been top to bottom, and they’ve been very welcoming to talk about what they find frustrating, what they find exciting, what they want to do and the conversations they want to have. They’re trying to engage us as peers, as scientists, rather than as consumers.” —Dr. Ayre, podcast interview
“They have been open about things that don’t work and that from the corporate perspective aren’t spit polished. They’re willing to show rough edges on things...That speaks to a bit of integrity.” —Dr. Ayre, podcast interview
“The key behind any successful company, is really the people. And the thing that has come across to me very strongly, with everybody I’ve met, is a commitment.” —Dr. Frame, documentary
“...I’ve found [at STEMCELL] a very high rigor of quality control and quality checks... I’ve been really impressed by the number of controls and the types of controls that they’ve introduced to try and reduce variability. It really comes to a value within the company to generate consistent and reliable products.” —Dr. Stone, podcast interview
“What I’ve had from here is an appreciation of the amount of work that goes into a company to produce something… It gives me ideas of how I can hold other companies to account, to the accountability of their practices, their quality control.” —Dr. Frame, podcast interview
“Having somebody tell me that they’re interested in finding out what they can do to support me as a person, support me as a customer, support me as a colleague—that resonates with me.” —Dr. Ayre, documentary
“The role that STEMCELL has as an employer, can’t be underestimated or taken for granted. The postdoctoral community feels, rather, that they’re getting pushed out of academia. There doesn’t seem to be a future, just as being a scientist. So the philosophy behind STEMCELL gives me some hope that postdocs can be seen as valuable people, and potentially a valued employee.” —Dr. Frame, documentary
“STEMCELL is doing things that they don’t necessarily have to do... [they are] a successful company that produces good quality products and they could just stop at that but... they do want to reach out to scientists in terms of getting the publications out there and the latest methods out there, really to move the field forward.” —Dr. Frame, podcast interview
“Any way that we can make the information overload easier to handle and digest is really helpful to the scientific community. That’s one way that STEMCELL is reaching out.” —Dr. Frame, podcast interview
“Looking at it from a Scientists Helping Scientists aspect, I do think that they are trying to be the best support system that they can.” —Dr. Stone, podcast interview
“I’ve seen that Scientists Helping Scientists is a legitimate statement about the company culture.” —Dr. Ayre, documentary
“It does all start from the top… the philosophy and culture comes from Allen. He knows what they do, he knows what they’re good at, he knows the vision... It is the people that make this company. They have the same values, the vision, the strategy, the goals. It’s cohesive. There are separate groups, but they all talk to each other. It doesn’t feel splintered in any way.” —Dr. Frame, podcast interview
“Something that I found striking about the company is the mobility of people within the company. Many people come through R&D and then move on to Marketing, Sales, or Technical Support. Those people who moved into those roles are originally bench scientists. They have been there... and they know what the troubles are. They stay and are retained and are moved around the company to fill in these other roles because they believe in what STEMCELL’s doing and how they are doing it. If there wasn’t the integrity of scientists throughout the company, I don’t think that they would have the kind of employee retention that they see, which is very high.” —Dr. Stone, podcast interview
“They know the cool science that’s going on and they have the corporate culture that’s designed to support it.” —Dr. Ayre, podcast interview
“They have also shown us that profits get fed back into the company for R&D and product improvement...There aren’t any shareholders. As someone who is funded by charity, that’s something that sits well with me... Clearly what they are doing is working because people must be buying their products otherwise the company wouldn’t be growing like it is.” —Dr. Frame, podcast interview
“And they showed us those numbers [on profit reinvestment].” —Dr. Ayre, podcast interview
According to the feedback shared during the discussions as well as during the podcast, the panelists generally liked what they saw during their visit. Going forward, they want to see STEMCELL maintaining what we do well, particularly by ensuring the quality of our products. Indeed, the issue of reproducibility in science and the role that reagent consistency has in experimental reproducibility was at the forefront of Dr. Stone's mind when she came to review STEMCELL.
How can STEMCELL contribute to standardizing reagents and protocols across time, across distance, such that if I do an experiment today and repeat that experiment in 5 years in a different lab, I’m going to see the same result. That is really what I’m interested in, because I think it’s a true, achievable step that a company can accomplish to really address the reproducibility crisis.
Dr. Stone, podcast interview
Product quality is always a top priority at STEMCELL and we were pleased that the peer reviewers were able to see for themselves the rigorous quality control processes we have in place to ensure the reliability of our reagents. Dr. Stone's conclusion during the podcast on our approach to reproducibility was that “STEMCELL is doing the right thing and moving in the right direction." On a related note, we are committed to the continual improvement of our processes and, as Dr. Ayre mentioned in the podcast, are investing in the construction of a new Good Manufacturing Practices (GMP) facility to reach the higher regulatory compliance standards required to produce tools for use in clinical trials.
Accept, with Minor Revisions
The official Peer Review decision on whether STEMCELL operates true to its motto of “Scientists Helping Scientists”, delivered during the podcast, was a unanimous “ACCEPT, with minor revisions”. As noted in the podcast, the post-review discussion was not limited to positive aspects, but included critical observations. The peer reviewers also left with three key suggestions for improvement, which are summarized in Table 2 below.
Table 2. The review panel had three main suggestions for STEMCELL’s continued improvement
“I think that their biggest problem and what they’re struggling the most with is really connecting with the research user. They’re good at communicating through their website, in person at conferences, and through their sales reps, but that doesn’t really represent the person using that product.” —Dr. Stone, podcast interview
“They are trying to have a large footprint but they need to make sure that they maintain that connection with individuals who use their products.” —Dr. Stone, podcast interview
“I want STEMCELL to start finding its voice in trying to push science in particular directions. To raise the standard of discourse or to facilitate more conversations about the way that science is done.” —Dr. Ayre, podcast interview
“I think that STEMCELL might be growing faster than they realize… There has to be some sort of corporate push [to make sure that the company retains the values that started the company when it was small].” —Dr. Ayre, podcast interview
“It depends really on that leadership below Allen [Eaves] that’s going to make his goals of keeping the culture and small company feel happen and going to make STEMCELL either remain a trusted company that you can go to and rely on, or a company that used to be good but is no longer up to the standard.” —Dr. Stone, podcast interview
Our Response to the Reviewers
We are taking the reviewers' valuable feedback to heart and are working on their recommendations. To foster closer relationships with the researchers using our products, we are promoting transparency and trust. This peer review exercise was the first step in demonstrating our authenticity and we hope that it will generate useful discussion and feedback from the scientific community. We will also continue to provide the exceptional customer and technical support that we have become known for; putting scientific progress over profits when working with researchers to find the right tool for their needs. As Dr. Frame noted in the documentary, communication is a two-way street and we need to find better ways to connect with our end-users. We're taking action by creating more opportunities for researchers to comment, provide feedback or simply ask a question.
Dr. Ayre's major recommendation was for STEMCELL to use its voice to push the dialogue on science. Some of the ways we are promoting scientific discourse is by highlighting current challenges in the research fields near and dear to us, to help spread awareness and work with the scientific community to overcome them. We are also shining a spotlight on the demanding world of science that researchers face and have compiled resources to help scientists “work smarter, not harder". As Dr. Ayre noted, these are discussions that are needed to help push science forward. They are also discussions that cannot be held in a vacuum and require voices from the scientific community.
- How do you assess the quality of your pluripotent stem cell cultures?
- What is your experience working with organoids?
- How can researchers better overcome the demands of scientific research?
- What other challenges and issues should we bring to light?
Finally, we and the peer reviewers all agree that one of our major assets lies in our people and our company culture of being Scientists Helping Scientists. We are committed to maintaining our identity as we continue to expand. We are being careful to build our company around diverse, talented and driven people with a shared passion for science. By creating jobs, we are providing young scientists and researchers with employment options outside of academia, either in R&D or in a corporate setting, so they can give back to the research community and embody Scientists Helping Scientists. Finally, we are maintaining the reinvestment of our profits into Research & Development to create and improve upon the tools that scientists need to conduct their research with confidence. Like Dr. Frame, we want researchers who buy our products to feel confident that their valued research funding dollars are going towards research and development, not shareholders.
Overall, we and the peer reviewers concluded that the peer review exercise was a big step in the right direction towards transparency and open communication between STEMCELL and the research community. We humbly receive their verdict of “Accept, with minor revisions" and are striving to improve based on their recommendations. We are certainly open to their suggestion that there should be a follow up peer review exercise, which we think will help us continue to improve. Furthermore, in sharing the results of the peer review exercise with the scientific community at large, we are looking for your feedback. Visit our website to learn more and let us know how you think we are doing. How would you improve the peer review in the future? What questions would you have asked? We want to hear from you.
Appendix: The Peer Review Committee
Dr. Amy Stone, Senior Fellow, University of Washington
Dr. Stone is studying how cells communicate that they are infected with a virus. Specifically, she is trying to understand how macrophages sense RNA virus infections, what changes in polarization and function occur in those macrophages, and the downstream effects on the adaptive immune system.
Reason she is a scientist:
Dr. Stone chose a career in science because she wants to know how things work. She loves exploring new ideas and challenging the ways that we think about biological processes. She also loves to share her knowledge with others and teach them how to answer their own questions.
Motivation for participating in the peer review:
“I want to understand what drives STEMCELL's discovery, directions, and products. I further want to validate the controls, quality assurances, and development of STEMCELL's products. This peer-review committee allows me to ask deep questions about why STEMCELL puts its efforts in the direction it chooses, how each product is evaluated and quality is assured for every lot that is shipped to scientists, and to report these findings back to my peers in an unbiased, truthful approach. With my background in industry and my current position in academia, I have the perfect perspective to observe, evaluate, and communicate the methods/practices of STEMCELL to academics. Finally, I want to be part of this peer-review committee to contribute in a meaningful way to resolve the reproducibility crisis currently plaguing bioscience. By forming this committee and opening themselves up for peer review, STEMCELL is addressing the reproducibility crisis and saying,"we want to be as rigorous, consistent and accurate as we possibly can be, and we want the best minds helping us do this". As a scientist, I can only be as accurate as my tools. This committee allows myself and the entire scientific community to answer the question of "Can I rely on STEMCELL's products to generate reproducible, accurate, trustworthy data?" I think that question and the steps that STEMCELL is taking to address that question in forming this committee is admirable and I want to be a part of this. Let other companies take note and follow STEMCELL's lead in addressing these important challenges."
Dr. Craig Ayre, Postdoctoral Fellow, Atlantic Cancer Research Institute
Dr. Ayre is working on extracellular vesicles: how they form, what their underlying functions are, why cells release them. Beyond understanding their basic biology, he's interested in finding ways to use extracellular vesicles to benefit the future of precision medicine.
Reason he is a scientist:
Science gives Dr. Ayre a way to ask questions and find answers to things that at one time, he didn't even know were questions. He's driven by the knowledge that through hard work and application, he might get to add “just a tiny new sliver of knowledge to the world".
Motivation for participating in the peer review:
“The nature of science is changing. Once upon a time, science was presented as two camps - the academics who pursue "pure" research, and the business of science for exploiting this knowledge. That isn't the case anymore. More of us belong in this amorphous middle of not quite academia, not quite industry where we dabble in basic and applied science in a far more fluid spectrum. STEMCELL is the largest biotechnology company in Canada, and so has a major "industry" footprint. At the same time, STEMCELLl is also leading research in areas of cell biology, that facilitate the work of many others. Getting insight in how this dynamic functions, and on what one of the heavy hitters of biotechnology in Canada views as their future, is a valuable opportunity. It's also important that the notion of peer-review be applied outside of academics. Industrial "company" science isn't the alternative career anymore and it needs to be given the same gravitas and platform to demonstrate its rigor and benefit to research as any other research program. I would appreciate the ability to participate in that process."
Dr. Fiona Frame, Postdoctoral Research Associate, University of York
Dr. Frame is working on primary epithelial prostate cells derived from benign and cancerous patient tissue. She uses these cells as a model to test current and novel treatments to assess consequences of treatment and therapy resistance mechanisms.
Reason she is a scientist:
Dr. Frame has been captivated by “the science of life!" from a young age. She is fascinated by the mechanisms and processes of life that can go wrong and result in disease. As a scientist, she wants to contribute to knowledge and be part of the solution.
Motivation for participating in the peer review:
“I think it is a marvellous opportunity to find out what goes on inside a company, how the people in the company think, how new reagents are developed and what the company is doing to provide what scientists need. Being part of a committee would be fascinating because of the mixture of people that would come together. New perspectives and exchange of knowledge can only help everybody's research progress. Sometimes we get a little stuck in our lab with our methods and it is of tremendous value to step out of the environment and be exposed to new ideas and different ways of doing things. Typically we purchase things from companies and either it works or doesn't and the feedback can be reasonably good or fairly minimal. This peer-review committee feels like something unique. I would be delighted to be part of this process, which can only ultimately benefit the company and the scientific community that it serves.”