Consider the following two URLs:
jar:file:/test.jar!/BOOT-INF/classes!/foo.txt
jar:file:/test.jar!/BOOT-INF/classes/foo.txt
They both reference the same foo.txt file in the BOOT-INF/classes
directory of test.jar, however the first URL does so via the
nested BOOT-INF/classes archive. Previously, this difference in the
URLs would lead to PathMatchingResourcePatternResolver returning two
resources for foo.txt when asked to find all resources matching the
pattern classpath*:/**/*.txt.
This commit updates our Handler that is used for jar: URLs to consider
the two URLs above to be equivalent such that url1 is equal to url2
and the two urls will produce the same hash code.
Closes gh-7449
Previously, if Boot's JarURLConnection pointed to the root of a nested
entry, e.g. /BOOT-INF/classes, a call to getInputStream() would throw
an IOException. This behavior is reasonable for a URL that points
to the root of a normal jar as the jar itself is on the class path
anyway. However, for a nested jar it meant that a call to
ClassLoader.getResources("") would not include URLs for any nested
jars and directories (/BOOT-INF/classes and jars in /BOOT-INF/lib).
This is due to some logic in URLClassPath.Loader.findResource that
verifies a URL by opening a connection and calling getInputStream().
The result of missing URLs for the root of nested jars and directories
is that classpath scanning that scans from the root (not a good idea
for performance reasons, but something that we should support) would
not find entries in /BOOT-INF/classes or in jars in /BOOT-INF/lib.
This commit updates our JarURLConnection so that it no longer throws
an IOException when asked for an InputStream for the root of a nested
entry (directory or jar).
Fixes gh-7003
This commit restores the logic in Handler that was changed when
d20ac56a was merged, while leaving the structural improvements intact.
In addition to a couple of changes where a typo meant the wrong
variable was being referenced, some logic branches now return false
rather than called super. This realigns our Handler's behaviour with
that of the JDK's.
Some more tests have also been added to try to catch the problems that
were introduced during the merge.
Closes gh-7021
Previously our handler didn't override parseURL or sameFile which
resulted in behaviour that differed from that of the JDK's handler.
Crucially, this would result in our JarURLConnection being passed
a spec that didn't contain a "!/". A knock-on effect of this was
that the connection would point to the root of the jar rather than
the intended entry.
Closes gh-7021
URL.getContent() is shorthand for URL.openConnection().getContent().
It creates an InputStream that isn't explicitly closed. This means
that a file handle remains open until the URLConnection is garbage
collected. This can lead to the process exceeding the limit for open
files.
Previously, LaunchedURLClassLoader was using getConent() when
proactively defining a package for a class that is about to be loaded.
getContent() was used to access nested jar files to check if they
contained the package and, if so, to retrieve the jar's manifest.
In place of using getContent(), this commit uses JarURLConnection's
getJarFile() method which provides access to the JarFile without the
unwanted side-effect of opening an input stream.
Closes gh-7180
Previously, RandomAccessDataFile used a semaphore and acquired it
interruptibly. This meant that an interrupted thread was unable to
access the file. Notably, this would prevent LaunchedURLClassLoader from
loading classes or resources on an interrupted thread.
The previous commit (937f857) updates RandomAccessDataFile to acquire
the semaphore uninterruptibly. This commit adds a test to
LaunchedURLClassLoader to verify that it can now load a resource from
an interrupted thread.
Closes gh-6683
This commit improves the performance of JarURLConnection. There are two
main changes:
Firstly, the way in which the spec is determined has been changed so
that it’s no longer necessary to create an absolute file. Instead,
the JarFile’s pathFromRoot is used to extract the spec from the URL
relative to the JarFile.
Secondly, the number of temporary Objects that are created has been
reduced, for example by tracking an index as we process a String
rather than creating a new substring for each iteration.
See gh-6215
JarURLConnection is very performance sensitive. The change in 3772d9f
meant that every JarURLConnection would create a FilePermission,
irrespective of whether it was actually used.
This commit updates JarURLConnection to create its FilePermission
lazily. When there is no security manager a permission will no longer
be created at all.
Closes gh-5411
See gh-6215
Previously, JarURLConnection assumed that that URL with which it was
created would contain the absolute path of the underlying jar file.
This meant that when it was created with a relative URL, it could fail
to find an entry or throw a StringIndexOutOfBoundsException.
This commit updates the logic for normalizing the input URL so that
both absolute and relative URLs are supported.
Closes gh-6109
Previously, JarURLConnection didn't override getPermission(). This
meant that it required all permissions. This was at odds with the
Oracle JVM's concrete sun.net.www.protocol.jar.JarURLConnection which
overrides getPermission to return a FilePermission with the read
action for the path of the underlying jar.
This commit updates our JarURLConnection to align its behaviour with
sun.net.www.protocol.jar.JarURLConnection.
Closes gh-5411
Using a separate thread to call the application's main method is
unnecessary – the context class loader of the current thread can be
updated instead – and makes exception and exit code handling more
complicated than it needs to be.
This commit updates the Launcher so that it calls the main method
runner using the current (main) thread. As a result, any exception
that's thrown will be caught by the JVM and result in a non-zero exit
code being returned from the process.
Closes gh-5922
Update CentralDirectoryParser to reduce the number of objects created
when parsing the central directory. A single CentralDirectoryFileHeader
object is now reused as entries are parsed.
Fixes gh-5260
Previously, the CLI would always use the class loader to try to locate
a source file. This was contrary to the SourceOptions javadoc which
states that the class loader is “used to try and load files that are
not found in the local filesystem”. This provide to be problematic on
Windows when a fully-qualified source file was supplied. The driver
letter colon slash (e.g. c:/) at the start of the path is considered
invalid for a class path resource by URLClassPath.Loader resulting in an
IllegalArgumentException being thrown.
A workaround for this URLClassPath behaviour was added in a71c9b5d. It
was removed as part of reworking LaunchedURLClassLoader to use a
conventional delegation model in 87fe0b2a. It was then reinstated in
cc140b2c. This work around is undesirable as it causes
LaunchedURLClassLoader’s behaviour to diverge from URLClassLoader’s
behaviour (this is contrary to the comments in the test added in
a71c9b5d which incorrectly tests the two class loader with different
class paths. If the two class loaders are created with the same class
path then their behaviour is the same).
This commit updates SourceOptions to make its behaviour match its
javadoc so that a search of the class path is only performed if the
filename doesn’t exist on the filesystem. Furthermore, when running on
Windows, if the filename is an absolute path no further searching is
performed as the path cannot reliably be used to search the class path
due to the behaviour of URLClassPath.Loader when given a path that
is an absolute file path on Windows. This ensures that the user is
presented with an error message indicating that the file could not be
found.
Closes gh-5650
Previously, JarURLConnection would fail when created with a URL that
began with jar:file:// as the double-slash is not included in jarFile.getUrl().getFile().
This commit updates JarURLConnection to normalise the value return from
url.getFile() to remove a double-slash when present.
Fixes gh-5287
Closes gh-5289
Previously, JarLauncher considered any entry whose name began with
BOOT-INF/classes/ as being a nested entry. This was incorrect as it
meant that subdirectories of BOOT-INF/classes/ would be added to the
classpath rather than just BOOT-INF/classes/ itself.
This commit updates JarLauncher so that only directory entries with a
name equal to BOOT-INF/classes are used.
Closes gh-5610
Previously, if a JarFile was created from a directory nested inside
another jar file, it would look for the manifest in
pathFromRoot/META-INF/MANIFEST.MF. This is incorrect as, unlike a
JarFile created from a jar file, the archives are one and the same
so the manifests should be too.
This commit updates JarFile so that its aware of how it was created
(direct from a file, from a nested directory, from a nested jar). If
it was created from a file or from a nested jar, it uses its manifest.
If it was created from a nested directory, it uses the manifest of the
root archive.
Closes gh-5609
Previously, if loader.path directly specified a jar file that contained
nested archives (.zip or .jar), launching would fail unless those
nested archives were uncompressed. However, if loader.path specified a
directory that contained such a jar file the launch would succeed. This
was because the nested archives within the jar were ignored.
This commit updates PropertiesLauncher so that its behaviour in the
scenarios described above is consistent by not looking for archives
nested with a jar that’s be specified on loader.path. The javadoc for
loader.path has also been updated to make it clear that loader.path
can points to directories or jar files, bringing it into line with
the reference guide.
Closes gh-3701
Following changes to LaunchedURLClassLoader made in 87fe0b2a, it is
no longer necessary for the launcher to load MainMethodRunner via
reflection as both the app class loader that the launcher URL class
loader share the same MainMethodRunner class.
This commit takes advantage of this by updating Launcher to instantiate
MainMethodRunner directly rather than via reflection, removing one
source of possible exceptions in the launcher.
As the MainMethodRunner is now loaded directly and its class is shared
between the two class loaders, there’s no longer a need for it to
implement Runnable. This allows it to throw Exception from its run
method, rather than having to wrap any Exception in a RuntimeException.
Lastly, rather than catching any exception thrown from the launch,
Launcher and its subclasses have been updated to allow this exception
to be thrown from the main method. This allows the Exception to reach
the JVM, to be processed by our registered uncaught exception handler,
and to trigger the JVM’s standard processing for exiting due to a
failure. This removes the need for the Launcher itself to call
System.exit(1) and ensures that the exception is only output to the
console if it hasn’t been registered as a logged exception.
Closes gh-5358
Previously, when defining a package for a class, LaunchedURLClassLoader
would use the manifest from the first location that contained the
required package. If the package was split across multiple locations,
this could lead to the manifest from a jar other than the one that
contains the class being used.
This commit updates LaunchedURLClassLoader so that it will use the
manifest of the jar file that contains the class which triggered the
definition of the package.
Closes gh-5485
LaunchedURLClassLoader preemptively defines the package for any
classes that it attempts to load so that the manifest from a nested
jar is correctly associated with the package. This can lead to a race
where the package is defined on two threads in parallel, resulting
in an IllegalArgumentException being thrown.
This problem was manifesting itself as a NoClassDefFoundError.
If the initialization of a class failed due to the above-described
IllegalArgumentException, subsequent attempts to use that class
would then fail with a NoClassDefFoundError.
This commit updates LaunchedURLClassLoader to catch the
IllegalArgumentException and then double-check that the package has
already been defined. This approach, including thrown an
AssertionError when the second check fails, is modelled on the
approach taken by URLClassLoader.
Closes gh-5464
Previously, the Launcher would call ex.printStackTrace for any
exception that was thrown when launching the application. This was
unnecessary as the stack trace should already have been logged by
the application when it failed to start.
This commit removes the call to ex.printStackTrace, thereby allowing
an exception to be logged only once or not at all after successful
failure analysis.
Closes gh-5358
Previously, JarURLConnection would corrupt a URL that contained a
mixture of encoded and unencoded double-byte characters. URLs that
only contained unencoded double-byte characters were not affected as
they are passed through as-is.
This commit updates JarURLConnection.JarEntryName to correctly handle
characters with a value that won't fit in a single signed byte (a
value greater than 127). Such characters are now URL encoded and then
written to the output stream as multiple bytes.
Closes gh-5194
When an application is run as an executable archive with nested jars,
the application's own classes need to be able to load classes from
within the nested jars. This means that the application's classes need
to be loaded by the same class loader as is used for the nested jars.
When an application is launched with java -jar the contents of the
jar are on the class path of the app class loader, which is the
parent of the LaunchedURLClassLoader that is used to load classes
from within the nested jars. If the root of the jar includes the
application's classes, they would be loaded by the app class loader
and, therefore, would not be able to load classes from within the
nested jars.
Previously, this problem was resolved by LaunchedURLClassLoader being
created with a copy of all of the app class laoder's URLs and by
using an unconventional delegation model that caused it to skip its
parent (the app class loader) and jump straight to its root class
loader. This ensured that the LaunchedURLClassLoader would load both
the application's own classes and those from within any nested jars.
Unfortunately, this unusual delegation model has proved to be
problematic. We have seen and worked around some problems with Java
Agents (see gh-4911 and gh-863), but there are others (see gh-4868)
that cannot be made to work with the current delegation model.
This commit reworks LaunchedURLClassLoader to use a conventional
delegate model with the app class loader as its parent. With this
change in place, the application's own classes need to be hidden
from the app class loader via some other means. This is now achieved
by packaging application classes in BOOT-INF/classes (and, for
symmetry, nested jars are now packaged in BOOT-INF/lib). Both the
JarLauncher and the PropertiesLauncher (which supports the executable
jar layout) have been updated to look for classes and nested jars in
these new locations.
Closes gh-4897
Fixes gh-4868
Previously, the Launcher was creating a new runner thread that would
call the application's main method. An exception thrown by this thread
is handled differently to one thrown by the JVM's main thread leading
to different exit behaviour. Furthermore, the separate thread isn't
actually necessary.
This commit removew the use of a separate runner thread from the
Launcher. This means that the JVM's exit behaviour will be consistent
and also removes the overhead of createing a starting an extra thread.
Closes gh-5006
Ensure that JarFile caches are cleared once the ApplicationContext has
loaded. Caches are cleared manually with the assumption that no
further class loading is likely.
Closes gh-4882
Refactor `spring-boot-loader` to reduce the amount of memory required
to load fat & exploded jars. Jar files now no longer store a full list
of entry data records, but instead use an array of entry name hashes.
Since ClassLoaders often ask each JAR if they contain a particular
entry (and mostly they do not), the hash array provides a quick way to
deal with misses. Only when a hash does exist is data actually loaded
from the underlying file.
In addition to the JarFile changes, the Archive abstraction has also
been updated to reduce memory consumption.
See gh-4882
Remove the use of JDK loggers in Launcher and PropertiesLauncher to
ensure allow the custom Log4j2 `LogManager` to be used with Spring Boot
applications.
Fixes gh-3815
Previously, if an application’s main method threw an exception,
MainMethodRunner would catch the exception and call System.exit(1).
This meant that the JVM would exit, irrespective of whether or not
any non-daemon threads were running. In contrast, when an application’s
main method was invoked directly (in an IDE, for example) the JVM
would not exit if one or more non-daemon threads were running. This
is standard JVM behaviour that we should be consistent with in the
launcher.
This commit updates MainMethodRunner to wrap any exception thrown by an
application’s main method in a RuntimeException and rethrow it. This
alllows the JVM to handle the exception and use its normal rules for
deciding whether or not it should exit.
Closes gh-4984
PropertiesLauncher creates a ClassLoader that is used by the Launcher
to load an application’s classes. During the creation of this
ClassLoader URLs from its ClassLoader. This resulted resulting in Java
agents that are added to the system class loader via the -javaagent
launch option being available on both the system class loader and the
created class loader. Java agents are intended to always be loaded by
the system class loader. Making them available on another class loader
breaks this model.
This is the same problem that was in ExecutableArchiveLauncher and
that was fixed in ee08667e (see gh-863).
This commit updates PropertiesLauncher so that it skips the URLs of
any Java agents (found by examining the JVM’s input arguments) when
copying URLs over to the new ClassLoader, thereby ensuring that Java
agents are only ever loaded by the system class loader.
Closes gh-4911
This commit completes the changes to consistently used static final
fields for Log instances that were started in ec2f33f9. Specifically it:
- Removes this. when accessing logger fields that are now static
- Renames some fields from log to logger
- Makes some logger fields static
See gh-4784
Update exit code support to allow the ExitCodeGenerator interface to
be placed on an Exception. Any uncaught exception implementing the
interface and returning a non `0` status will now trigger a System.exit
with the code.
Fixes gh-4803
Some libraries like aspectj are using findResource to see the raw
bytecode of a class. It will even call findResource for every method of
every class of beans that are post processed. This can be significant
performance hit on startup when LaunchedURLClassLoader and there are a
lot of nested jars.
See gh-3640
Fixes gh-4557
Some libraries like aspectj are using findResource to see the raw
bytecode of a class. It will even call findResource for every method of
every class of beans that are post processed. This can be significant
performance hit on startup when LaunchedURLClassLoader and there are a
lot of nested jars.
See gh-3640
Fixes gh-4557
Previously, JarFileArchive would always unpack any entries marked for
unpacking to ${java.io.tmpdir}/spring-boot-libs. This could cause
problems if multiple Spring Boot applications were running on the same
host:
- If the apps are run as different users the first application would
create the spring-boot-libs directory and the second and subsequent
applications may not have write permissions to that directory
- Multiple apps may overwrite each others unpacked libs. At best this
will mean one copy of a jar is overwritten with another identical
copy. At worst the jars may have different contents so that some of
the contents of the original jar disappear unexpectedly.
This commit updates JarFileArchive to use an application-specific
location when unpacking libs. A directory beneath ${java.io.tmpdir} is
still used but it's now named <jar-file-name>-spring-boot-libs-<uuid>.
A loop, limited to 1000 attempts, is used to avoid problems caused by
the uuid clashing.
Closes gh-4124
When nested jars are being used, hasMoreElements requires opening a
connection for an entry in every nested jar. If that entry doesn't
exist, a FileNotFoundException is thrown to indicate that a particular
jar doesn't contain the requested entry. This exception is used to
indicate the lack of an entry and is then swallowed, i.e. its stack
trace is of no importance. This means that the performance of
hasMoreElements can be improved by switching on fast exceptions while
it's being called. When fast exceptions are switched on a general
purpose pre-initialized FileNotFoundException is thrown rather than
creating a new FileNotFoundException instance each time.
In certain situations, the use of fast exceptions as described above
can improve performance fairly significantly. The JRE's default SAAJ
implementation uses META-INF/services-based discovery for _every_
request that's handled by Spring Web Services. Each discovery attempt
results in hasMoreElements being called making its performance
critical to throughput.
See gh-3640
Phillip Webb
Andy Wilkinson
Johnny Lim
Spring Buildmaster
dreis
Stephane Nicoll
hengyunabc
Vladimir Tsanev
Kirill Vlasov
Ilayaperumal Gopinathan
Martin Lippert
Dave Syer