test: increase number of tries to find tz not near midnight#4756
Conversation
|
Lets talk through again what the random selection (rather than a deterministic selection of known based on well-known timezone names and the offset of current) is buying us. Is it only the chance of early discovery of the effect of an unusual (or broken in the library) timezone that some user might be in (off by |
|
I think the main thing is that it avoids baking in dependencies like we're seeing now, where running tests in the hours between UTC midnight and PST8PDT midnight cause test failures that we don't find until later because our pull requests are mostly generated in western hemisphere working hours. We might do better to have it generate a synthetic time zone with a random UTC offset in the window we want. That would avoid the testing-the-library misfeature. |
This addresses a failure of the automated tests that occurred when testing the
timezone_not_near_midnight()helper method.The test operates by mocking the list of timezones available to 3 that should be rejected because they are close to midnight and 1 that should be selected. The test is repeated 5 times to limit the chance that the method could pass the test by picking one of its options at random.
It turns out that the infinte-loop prevention in
timezone_not_near_midnight(), which allowed up to 20 guess-and-check attempts, leads to an expected failure of this test about once every 50 runs. That comes from1 - (1-(3/4)**20)**5 ~ 0.015as the probability of at least one of the 5 tests failingThis PR increases the number of attempts to 50 which should give a test failure every 10^5 tries or so.
None of this should affect use of the method itself, since about 11 of 12 real time zones will be outside the 2-hour rejection window. It's unlikely ever to need more than a few tries.