Pygeos option for geopandas slows down read_file procces

I have a 95000 item shapefile, I import the file with GeoPandas using read_file . The item are polygons with fairly simple geometry, with the biggest polygon with 316 points.

Problem

While doing some speed tests using the option options.use_pygeos = True actually slows down the read file process.

I am using GeoPandas version 0.10.2 and pygeos version 0.12.0. I have read that someone had the same problem while upgrading Geopandas from 0.8.2 to 0.9 in this post Geopandas performance loss when updating from version 0.8.2 to 0.9

The actual slow down :

time with pygeos 10.6 s
time without pygeos 5.3 s

Question

Has anybody reproduce this behavior?, how can we avoid this?