•  


Unstable results when running the same query twice · Issue #4627 · meilisearch/meilisearch · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable results when running the same query twice #4627

Closed
Kerollmops opened this issue May 10, 2024 · 2 comments
Closed

Unstable results when running the same query twice #4627

Kerollmops opened this issue May 10, 2024 · 2 comments
Labels
bug Something isn't working as expected

Comments

@Kerollmops
Copy link
Member

Describe the bug
When running the following query, the engine is returning unstable results. Note that everything is on the private Discord conversation with our customer .

{
    
"displayedAttributes"
: [
        
"
*
"

    ],
    
"searchableAttributes"
: [
        
"
title
"

    ],
    
"filterableAttributes"
: [
        
"
category
"
,
        
"
category_ids
"
,
        
"
channel
"
,
        
"
handle
"
,
        
"
language
"
,
        
"
made_for_kids
"
,
        
"
num_ab_tests
"
,
        
"
shorts
"
,
        
"
subscriber_count
"
,
        
"
upload_date
"
,
        
"
view_count
"
,
        
"
xf90
"

    ],
    
"sortableAttributes"
: [
        
"
upload_date
"
,
        
"
view_count
"
,
        
"
xf90
"

    ],
    
"rankingRules"
: [
        
"
words
"
,
        
"
typo
"
,
        
"
proximity
"
,
        
"
sort
"
,
        
"
exactness
"

    ],
    
"stopWords"
: [],
    
"nonSeparatorTokens"
: [],
    
"separatorTokens"
: [],
    
"dictionary"
: [],
    
"synonyms"
: {},
    
"distinctAttribute"
: 
null
,
    
"proximityPrecision"
: 
"
byWord
"
,
    
"typoTolerance"
: {
        
"enabled"
: 
true
,
        
"minWordSizeForTypos"
: {
            
"oneTypo"
: 
5
,
            
"twoTypos"
: 
9

        },
        
"disableOnWords"
: [],
        
"disableOnAttributes"
: []
    },
    
"faceting"
: {
        
"maxValuesPerFacet"
: 
100
,
        
"sortFacetValuesBy"
: {
            
"*"
: 
"
alpha
"

        }
    },
    
"pagination"
: {
        
"maxTotalHits"
: 
300000000

    },
    
"searchCutoffMs"
: 
null

}

To Reproduce
Unfortunately, we don't have a clear reproducer but running the above query multiple times.

Expected behavior
The results must be stable and sorted by xf90.

Meilisearch version:
v1.8.0

Additional context
Running on a dedicated Digital Ocean instance on the Cloud.

@Kerollmops Kerollmops added the bug Something isn't working as expected label May 10, 2024
@dureuill
Copy link
Contributor

dureuill commented May 13, 2024

A reason for result unstability introduced in v1.8 is the search cutoff.

I'm pretty sure this is what we're seeing.
To confirm, one can take a look at the score details looking for Skipped ranking rules.

If the behavior is undesirable, one should increase the search-cutoff-ms setting .

@Kerollmops
Copy link
Member Author

Kerollmops commented May 13, 2024

Thank you, @dureuill ??

You found the issue! This is only because the searchCutoffMs was set to 1500s, which stopped the computation and returned very non-relevant results. So, it is expected, and there is a workaround to increase the cutoff timing simply.

With the Default Cutoff

"_rankingScoreDetails"
: {
    
"words"
: {
        
"order"
: 
0
,
        
"matchingWords"
: 
1
,
        
"maxMatchingWords"
: 
1
,
        
"score"
: 
1.0

    },
    
"typo"
: {
        
"order"
: 
1
,
        
"typoCount"
: 
0
,
        
"maxTypoCount"
: 
1
,
        
"score"
: 
1.0

    },
    
"proximity"
: {
        
"order"
: 
2
,
        
"score"
: 
1.0

    },
    
"xf90:desc"
: {
        
"order"
: 
3
,
        
"value"
: 
258587.8854961832

    },
    
"skipped"
: {
        
"order"
: 
4

    }
}

With a High Cutoff Everything is Fine

"_rankingScoreDetails"
: {
    
"words"
: {
        
"order"
: 
0
,
        
"matchingWords"
: 
1
,
        
"maxMatchingWords"
: 
1
,
        
"score"
: 
1.0

    },
    
"typo"
: {
        
"order"
: 
1
,
        
"typoCount"
: 
0
,
        
"maxTypoCount"
: 
1
,
        
"score"
: 
1.0

    },
    
"proximity"
: {
        
"order"
: 
2
,
        
"score"
: 
1.0

    },
    
"xf90:desc"
: {
        
"order"
: 
3
,
        
"value"
: 
258587.8854961832

    },
    
"exactness"
: {
        
"order"
: 
4
,
        
"matchType"
: 
"
noExactMatch
"
,
        
"matchingWords"
: 
0
,
        
"maxMatchingWords"
: 
1
,
        
"score"
: 
0.16666666666666666

    }
}

Sign up for free to join this conversation on GitHub . Already have an account? Sign in to comment
Labels
bug Something isn't working as expected
Projects
None yet
Development

No branches or pull requests

2 participants
- "漢字路" 한글한자자동변환 서비스는 교육부 고전문헌국역지원사업의 지원으로 구축되었습니다.
- "漢字路" 한글한자자동변환 서비스는 전통문화연구회 "울산대학교한국어처리연구실 옥철영(IT융합전공)교수팀"에서 개발한 한글한자자동변환기를 바탕하여 지속적으로 공동 연구 개발하고 있는 서비스입니다.
- 현재 고유명사(인명, 지명등)을 비롯한 여러 변환오류가 있으며 이를 해결하고자 많은 연구 개발을 진행하고자 하고 있습니다. 이를 인지하시고 다른 곳에서 인용시 한자 변환 결과를 한번 더 검토하시고 사용해 주시기 바랍니다.
- 변환오류 및 건의,문의사항은 juntong@juntong.or.kr로 메일로 보내주시면 감사하겠습니다. .
Copyright ⓒ 2020 By '전통문화연구회(傳統文化硏究會)' All Rights reserved.
 한국   대만   중국   일본