By 苏剑林 | February 02, 2024
Since the launch of Cool Papers, many users have suggested adding a search function. Later, a simple page-level search was indeed implemented in the frontend using JavaScript, which solved the needs of some users. However, some readers still hope for a more complete global search. To be honest, I understand that this demand exists, but the data on Cool Papers is accumulated day by day. Since it has only been online for a month, the number of papers is not yet large, so building a comprehensive search engine wouldn't be very meaningful. Furthermore, building search is not my forte, and I haven't found a particularly good way to utilize LLMs to optimize search yet, among other reasons. All in all, for now, the conditions are not right to implement a comprehensive and unique search, so it’s better not to do it at all (though everyone is welcome to brainstorm in the comments section).
Later, after discussions with colleagues, I came up with a "borrowing a gift to present to Buddha" approach—writing a Chrome redirect extension that can redirect from any page to Cool Papers. This way, we can use any method (such as Google search or directly using the official Arxiv search) to find papers on Arxiv, and then with a simple right-click, jump to Cool Papers. This extension went live on the Chrome Web Store two weeks ago, and after some adjustments on the server side last week, it is now ready for everyone to try.
Extension Address: Cool Papers Redirector
The usage of the extension is very simple. After successful installation in Chrome, search for a paper using any method, then right-click on the corresponding location (sometimes even in a blank space). A "Redirect to Cool Papers" option will appear. Clicking this option will cause the browser to automatically detect a possible paper ID from the "selected text," "selected link," or "website address" (stopping as soon as one is detected) and automatically jump to the corresponding page on Cool Papers. The effect is as follows:
Right-click redirect effect 1
Right-click redirect effect 2
In fact, implementing such a redirect extension is quite simple, requiring only basic HTML and JS, provided by the dual guidance of GPT-4 and Kimi.
Since the development of the extension was not difficult, one major problem remained—the pressure brought by fully opening up access to historical papers. Many users have already noticed that previously Cool Papers could access specific papers via https://papers.cool/arxiv/[id], but this was limited to papers already in the database; otherwise, it would show "Not Found." If the use of the Cool Papers Redirector is to be popularized, it is inevitable that the crawling and access of all Arxiv historical papers must be opened up. Otherwise, if eight out of ten papers result in a "Not Found," the Redirector becomes almost meaningless.
To ensure that the main thread of Cool Papers—browsing the latest papers of the day—proceeds normally while opening up historical papers, I have adopted a multi-priority design for the Arxiv crawling queue and the Kimi dialogue queue (currently divided into three levels). First, "Super VIP" is for internal permissions, unlocked by entering a correct Magic Token, which is unrelated to ordinary visitors. Second is the priority for the current day's papers; when the time comes, the Arxiv queue prioritizes obtaining the list of the day's papers before processing historical paper requests. Similarly, the Kimi queue prioritizes today's papers, automatically "cutting in line" ahead of historical papers. Finally, the crawling and Kimi processing of historical papers are assigned the third priority.
In this way, I can basically ensure that the reading of the current day's papers is not affected, while using idle resources to process historical papers.
Compared to when "I wrote a paper-reading assistant website: Cool Papers" and "Happy New Year! Documenting the development experience of Cool Papers" were first published, after a month of improvements, Cool Papers has become much more refined in terms of functionality (though the interface remains as simple as ever). In addition to opening up access to all historical papers mentioned in this article, other changes include:
1. Added a bar at the bottom for searching papers, viewing/exporting reading records, etc. (Note: these functions are limited to the current page view);
2. Support for paper lists from specific dates. You can select a date via the calendar icon on the right side of the categories on the homepage, or by clicking the date text on the list page;
3. Switched to PDF.js for PDF previewing, supporting paper reading on mobile devices, while optimizing PDF text parsing to improve the quality of [Kimi] summaries;
4. The 4th button on the Bar allows you to switch [Kimi] to English output, making it convenient for international users or users who need to compare it with the original English text;
5. A whole bunch of minor bug fixes~
Overall, the interface doesn't look much different, but in reality, there are more or less improvements every day. Compared to the first version, the source code is now "unrecognizable." The next steps might include adding other paper sources, such as OpenReview, bioRxiv, etc. Please stay tuned!
This article shared a new way to open Cool Papers via a Chrome redirect extension and briefly reviewed the recent changes to Cool Papers.