#Japan #Tokyo #AI Search #NeCfru #file accessibility

Exploring File Accessibility: Insights from Necfru's Research

In a bold move to address the challenges of searching for files within organizations, Necfru, a Tokyo-based company, has released a comprehensive report titled “Unreachable Files.” This valuable document delves into the irritations many employees face when searching for files that are seemingly available but remain elusive due to various issues. It meticulously details how their product, necfru drive, measures the effectiveness of file searches, especially when incorporating AI-generated tagging methods.

Defining Unreachable Files

Necfru classifies files that cannot be found despite existence and access rights as “unreachable files.” The study aims to identify the scope of this issue and evaluate the effectiveness of different methods employed to solve it. The report highlights an array of industries including construction, broadcasting, public event management, and healthcare, all facing similar struggles when it comes to file accessibility.

Common queries arise such as, “Where can I find the plumbing diagrams from the building constructed 10 years ago?” or “What was the status of rights for this footage we produced?” These questions demonstrate the intrinsic issue of retrieving necessary information without having specific file names to refer to.

Evaluation Methodology

The research compiled an evaluation corpus of 138 files, representing various document types such as docx, pdf, png, jpg, xlsx, and pptx, totaling approximately 3.6 MB. Notably, 39% of these files were created in a way that their names did not reflect the content, contributing to the difficulty in locating them effectively.

The study posed 35 questions with predetermined correct answers, prohibiting any alterations or modifications during measurement to maintain integrity. Two search methodologies were then compared: the traditional keyword search focusing solely on file names versus a conversational AI search that incorporated AI-generated summaries and tags.

Insights from the Results

The findings are telling. The traditional keyword-based searches yielded correct results only 33.3% of the time for the top result, with 40% of searches failing to produce any relevant outcomes. In contrast, AI-enhanced conversational searches boasted a striking correctness rate of 96.7% for the top result, demonstrating the clear benefits of utilizing advanced technology in search queries.

Interestingly, while AI-generated tagging did not statistically outperform keyword searches (p=0.109), the quality of tags was deemed high, with an applicability rate of 98.2%. Even so, the report illustrates that better tagging alone does not inherently enhance searchability, as complex tagging issue persisted across various files.

Acknowledging Limitations

It’s essential to recognize the limits of this study conducted by Necfru itself. The small sample size (138 files) and specific task scenarios cannot be generalized to the broader operational landscape where files often number in the hundreds of thousands. Concerns regarding potential bias inevitably surface as the report's authors were involved in every aspect of its creation, from corpus design to reporting.

Despite these limitations, the researchers provided a transparent overview of results, including questions that yielded zero correct responses and issues where tags could not be used to generate search queries.

Concerns Regarding Search Efficiency

The report breaks down unreachable files into four categories, highlighting the common pitfalls experienced by users:

• - Mismatched Naming: Files that do not have relevant names.
• - Vocabulary Discrepancy: Different terminology used during file saving and searching.
• - Hidden Content: The needed information exists within the file but isn't easily found.
• - Lack of Determining Factors: Even when found, necessary permissions or specific usage cannot be confirmed.

By sharing both successes and failures openly, CEO Shunsuke Kusanagi emphasizes the necessity of using comprehensive data for informed decisions rather than merely presenting favorable outcomes. His approach encourages standard practices in corporate transparency, hoping to aid improved search engine functionalities moving forward.

Conclusion: A Call for Better Search Systems

Necfru's offering extends beyond mere data; it promotes a discourse around improving operational efficiencies and practical technology implementations. With constant flux in data organization, the revelations from “Unreachable Files” showcases both the promise and challenges that lie within the realm of AI-driven solutions and file retrieval processes.

As this landscape evolves, continuing to refine and test different methodologies will be crucial. Organizations should look towards an emergent future where searching for critical files does not have to feel like finding a needle in a haystack. For the full research report, visit Necfru's blog or access the complete data set on GitHub.