RFR [15] 8237909: Remove zipped index files feature

Pavel Rappo pavel.rappo at oracle.com
Tue Jan 28 18:41:00 UTC 2020


> On 28 Jan 2020, at 17:15, Hannes Wallnöfer <hannes.wallnoefer at oracle.com> wrote:
> Hi Pavel, 
> I fully concur with your reasoning for removing the jszip feature.

Removing JSZip is incidental here. It's the whole "zipped index files" feature
that is being removed.

> In script.js you left in the checks whether the *SearchIndex variables are undefined:
>      if (!moduleSearchIndex) {
>          ...
>      }
> These if-statements can be removed as the conditions will always be true (the vars are declared a few lines above but not assigned any value, so they will always be undefined). It’s ok to just invoke createElem(…) for all scripts/index files.

I will remove those conditional checks, thanks.

> I also wonder if there is any benefit left in creating the script tags dynamically vs. just adding them to the pages statically. I’m fine with keeping it that way for now, but maybe we could file another issue to check if there’s any reason to keep this over static <script> tags.
> Everything else looks good to me.
> Hannes

One benefit of creating those script elements dynamically seems to be the effectively
asynchronous transfer of the .js index files. Those files should be transferred
asynchronously, no question about that. We should make this explicit, though.

I'm not a web developer, but from what I've read there is more than one way to
load a script asynchronously. I wanted to muse over all the options properly in
the context of 8236935: Improve UX of the search control, where this can be
evaluated together with the different sorts of script loading events.


I intend to let this RFR sit here for at least 2 weeks. This, hopefully, will
make it accessible to more pairs of eyes. Maybe the original authors will chime in.
The perceived severity of this change should not be underestimated.


>> Am 28.01.2020 um 16:55 schrieb Pavel Rappo <pavel.rappo at oracle.com>:
>> Hello,
>> Please review the change for https://bugs.openjdk.java.net/browse/JDK-8237909:
>>   http://cr.openjdk.java.net/~prappo/8237909/webrev.00/
>> This change removes the "zipped index files" feature, which was introduced as
>> part of 8141492: Implement search feature in javadoc.
>> The "zipped index files" feature consists of generating the zipped index files
>> on the back end, and fetching & unzipping mechanics on the front end.
>> When documenting source files, the standard doclet accumulates index which is
>> later used by the JavaScript code serving the interactive search. The index
>> is written in two formats, .js (JavaScript) and .json (JSON). The latter is
>> then zipped.
>> When a browser accesses the pages using "http://" urls, the .zip index files are
>> transferred using XHR. Those files are then unzipped by the browser, using the
>> JSZip library, and parsed as JSON. If the transfer of the .zip index files fails
>> for whatever reason, the browser falls back on the alternative mechanism. This
>> mechanism transfers the .js index files by referring to them from dynamically
>> inserted <script src="... .js"> elements. Those files then are not additionally
>> parsed, as they are already data hardcoded in JavaScript code.
>> One of the reasons the .zip index files transfer may fail is using javadoc pages
>> in the "standalone" mode. When a browser accesses "file://" urls, there's no
>> HTTP server to send the XHR requests to. So the fallback mechanism kicks in and
>> the browser loads the .js index files instead.
>> Analysis
>> ========
>> From what I understand, the original intent was to reduce the transfer size of
>> the index files. The observations made during the recent upgrade of JSZip
>> (JDK-8236700) suggest that the feature is not working as intended. It is not
>> clear if it ever did. The proposal is to remove it for the following reasons:
>> 1. The feature in its current state does more harm than good (see JDK-8236922)
>> 2. Fixing, debugging, testing, and evolving require expertise beyond that of
>>   typical for the javadoc area
>> 3. The feature significantly complicates the front end and less so the back end
>>   code
>> 4. The feature relies on the 3rd party libraries, which require tracking &
>>   periodical upgrades
>> 5. The difference in size between the .zip and .js files is not that big (see below)
>> 6. The index files are transferred once and then used from cache
>> 7. Modern HTTP servers provide compression. This makes the net result
>>   virtually the same, compare:
>>                      | (current) js + zip, MB | (proposal) js files, MB
>>    ------------------+------------------------+------------------------
>>    no compression                        7.4                      5.8
>>    HTTP compression                      2.7                      1.4
>> Had this feature worked as intended, we would always transfer only the zipped
>> index files and the transfer size would not depend on whether the server uses
>> HTTP compression. But does this really outweigh the reasons stated above?
>> Summing all up. Removing the zipped index files feature will make the overall
>> interactive search feature (JDK-8141492) more robust. It will be less
>> complicated, have fewer dependencies (JSZip, JSZip Utils), and will push the
>> optimization down to HTTP.
>> Testing
>> =======
>> Here is how I tested this change.
>> 1. make clean && make docs
>> 2. Standalone test
>>    2.1. Opened the browser at file://...images/docs/index.html
>> 3. HTTP test
>>    3.1. Started an HTTP server at build/...images/docs
>>    3.2. Opened the browser at http://localhost...images/docs/index.html
>> Browser cache was cleared each time immediately before accessing the index.html page.
>> In both cases I checked that no zipped index files or the related JavaScript
>> libraries were accessed, and that the search worked as intended.
>> I also tried to access the resulting javadoc pages, served by an HTTP server on
>> my laptop, from a couple of mobile devices, all of which were on the same WiFi
>> network. Everything worked as intended.
>> Thanks,
>> -Pavel

More information about the javadoc-dev mailing list