Searching multiple words gives less results than it used to

Anca Luca I did the following tests on a fresh XWiki 18.2.0-SNAPSHOT:

• create 3 pages:
  • "Literature", with "poetry" in the content
  • "Science", with "gravity" in the content
  • "Science Literature", with "gravity poetry" in the content
• searching for any of the following queries: "literature science", "gravity poetry", "literature gravity", "science poetry" (without the quotes) includes all 3 pages in the results

So I can't reproduce your problem when there's no boolean operator in the query.

If I add the OR operator to the query then indeed only the last page that include both words is matched, same as with AND. This is unexpected for the end user, even though most users wouldn't use the OR operator. I don't know why this happens. My guess is that by adding the boolean operator the clause type changes (from optional to mandatory?) or the number of clauses changes ("literature OR science" is seen as a single clause rather than 2 clauses). This influence how "min should match" parameter works, so you get different results.

Note that the query is expanded. With debug mode on you can see that a query like "literature science" produces multiple clauses. This is probably the reason why we get all 3 pages in the results even though "min should match" parameter is set to 2. If I limit the query fields to "title" and I remove the configuration for "phraseFields", "bigramPhraseFields" and "trigramPhraseFields", but keep minShouldMatch set to 2, then I manage to reproduce your problem. In this case, a query like "literature science" produces a single result. BUT, using +(literature science) works for me: all 3 pages are matched.

After re-reading https://solr.apache.org/guide/solr/latest/query-guide/dismax-query-parser.html#mm-minimum-should-match-parameter I'd be in favour of setting "min should match" to 50%. Using a fixed value > 1, like we currently do, means there will always be cases when all tokens (clauses) must be matched, which is not what the user expects. Using a relative (percentage) value, ensures that the number of tokens that needs to be matched increases with the number of token you input. Matching at least half (rounded downwards) of the tokens seems like a good balance.

There is of course the option to stop setting "min should match" in the default configuration. I'm fine with that as well. It would help when you don't know precisely what you're searching for, and you input many tokens hoping at least one of them will be found. But it can also lead to lots of irrelevant results on a large wiki (most users won't know the Solr syntax to force that multiple tokens are matched).