Improve search functions

[Kixiron]

Closes #489, #156 and #13

• Use fuzzy searching, pattern matching and description text searching to get better search matches
• Fuzzy search gives better results and gives accurate rankings
• Description searching allows you to search for concepts as well as crates, so serialization will also pull up results for serde (With any better name matches before it)
• Also trimmed the string, so the leading/trailing whitespace thing ain't a thing

[jyn514]

Lots of comments! This already looks way better than the existing search though, thank you for tackling this :)

I would like to see a lot more tests added before this is merged - the search is now pretty complicated and I want to make sure this behaves as expected. Some things to test

• exact matches should always appear first
• crates that never had a successful build should not be shown
• no more than LIMIT crates are returned
• searching crate descriptions works
• searching regex shows results for regex-syntax
• searching Regex counts regex as an exact match
  • maybe we should just turn the query into all lowercase to start?
• searching 'redex' shows results for regex (fuzzy searching)

[Kixiron]

#708 brings up a good point, does the actual date a crate was published matter when we have semver?

[jyn514]

I don't know how to sort by semver in SQL. I think we should wait on that until later, we're making a big change already.

[jyn514]

Looks like you need to remove the migration to get the tests to pass: https://github.com/rust-lang/docs.rs/pull/721/checks?check_run_id=606111571#step:14:80

[jyn514]

You need to rebase over the changes to CI. Additionally, you may need to write an extra (small) dockerfile to add the fuzzy-search extension to postgres, see https://stackoverflow.com/a/54630526/7669110 for an example of what that would look like.

Past that, the current blocker is the performance of the query. The old query took less than half a second, this one takes almost 3. We've been working on this in Discord.

[Kixiron]

Got a much-improved query in, had to remove description searching, as with it the query took 700-800ms, while without it the query takes a consistent 300ms

[jyn514]

🎉 🎉 🎉