From uckelman at nomic.net Wed Dec 2 15:27:59 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Thu, 03 Dec 2009 00:27:59 +0100 Subject: locking in zipfs demo Message-ID: <20091202232759.F4229100D1@charybdis.ellipsis.cx> I'm puzzled by the ReentrantReadWriteLock used in ZipFilePath in various place. It seems to be locking calls to getResolvedPathForZip(), which sets the pathToZip member if it's null. Is there a reason for using a lock for this instead of making pathToZip volatile like the offsets ArrayList is? -- J. From Alan.Bateman at Sun.COM Thu Dec 3 00:51:00 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 03 Dec 2009 08:51:00 +0000 Subject: locking in zipfs demo In-Reply-To: <20091202232759.F4229100D1@charybdis.ellipsis.cx> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> Message-ID: <4B177BF4.5000604@sun.com> Joel Uckelman wrote: > I'm puzzled by the ReentrantReadWriteLock used in ZipFilePath in various > place. It seems to be locking calls to getResolvedPathForZip(), which > sets the pathToZip member if it's null. Is there a reason for using a > lock for this instead of making pathToZip volatile like the offsets > ArrayList is? > It looks like it was intended to support asynchronous close of the zip file system, in which case it should be using fs.begin() and fs.end() rather than its own lock (Rajendra - can you confirm this?). -Alan. From Alan.Bateman at Sun.COM Thu Dec 3 11:31:39 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 03 Dec 2009 19:31:39 +0000 Subject: Selector doesn't implement Closeable In-Reply-To: <4B094AF5.5030209@sun.com> References: <4B08347B.6000502@univ-mlv.fr> <4B094AF5.5030209@sun.com> Message-ID: <4B18121B.2080209@sun.com> Alan Bateman wrote: > : > I've created this bug to track it: > 6903753: (se) Selector should implement Closeable > R?mi - did you want to push this one and I can be reviewer? Alternatively, I can do it and you can be reviewer? -Alan. diff -r d5a1c012921d src/share/classes/java/nio/channels/Selector.java --- a/src/share/classes/java/nio/channels/Selector.java Sun Nov 29 15:24:32 2009 -0800 +++ b/src/share/classes/java/nio/channels/Selector.java Thu Dec 03 19:29:04 2009 +0000 @@ -25,6 +25,7 @@ package java.nio.channels; +import java.io.Closeable; import java.io.IOException; import java.nio.channels.spi.SelectorProvider; import java.util.Set; @@ -202,7 +203,7 @@ import java.util.Set; * @see SelectionKey */ -public abstract class Selector { +public abstract class Selector implements Closeable { /** * Initializes a new instance of this class. From alan.bateman at sun.com Thu Dec 3 11:44:23 2009 From: alan.bateman at sun.com (alan.bateman at sun.com) Date: Thu, 03 Dec 2009 19:44:23 +0000 Subject: hg: nio/nio/jdk: 2 new changesets Message-ID: <20091203194553.D4F3C41D71@hg.openjdk.java.net> Changeset: a2cbf3efeaa1 Author: alanb Date: 2009-12-03 19:36 +0000 URL: http://hg.openjdk.java.net/nio/nio/jdk/rev/a2cbf3efeaa1 More fixes to zip provider * s/pathForprint/pathForPrint/g * Fixed comment typos. * Use String ctor directly, instead of substring(), no need to keep larger array around. * Replaced StringBuffer with StringBuilder. * ZipFilePath.normalize() now works. * ZipPathParser.normalize() now chops trailing slashes. * ZipPathParser.resolve() now handles leading '..' in relative paths properly. * Introduced checkPath() to throw NPE and ProviderMismatchExceptions before using Paths passed as arguments. * Fixed endsWith() to handle absolute other path properly (/foo !endsWith /). * Fixed startsWith(): Absolute paths do not start with relative ones. * Simplified subpath(). (This was overcomplex.) * normalize() now returns null for paths equivalent to "." * Cannot relativize() when exactly one path is absolute. * getParent() returned null instead of "/" for single-element absolute paths ("/foo"). Contributed-by: uckelman at nomic.net ! src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java ! src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipPathParser.java Changeset: fa2afb1da3a0 Author: alanb Date: 2009-12-03 19:44 +0000 URL: http://hg.openjdk.java.net/nio/nio/jdk/rev/fa2afb1da3a0 Added unit test for zip provider path operations Fixed regression so that isDirectory returns true for directories ! src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipUtils.java + test/demo/nio/ZipFileSystem/Basic.java + test/demo/nio/ZipFileSystem/PathOps.java - test/demo/nio/ZipFileSystem/Sanity.java + test/demo/nio/ZipFileSystem/basic.sh - test/demo/nio/ZipFileSystem/sanity.sh From Rajendra.Gutupalli at Sun.COM Fri Dec 4 04:45:53 2009 From: Rajendra.Gutupalli at Sun.COM (Rajendra.Gutupalli@Sun.com) Date: Fri, 04 Dec 2009 18:15:53 +0530 Subject: locking in zipfs demo In-Reply-To: <4B177BF4.5000604@sun.com> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> Message-ID: <4B190481.2060709@Sun.com> Alan Bateman wrote: > Joel Uckelman wrote: >> I'm puzzled by the ReentrantReadWriteLock used in ZipFilePath in various >> place. It seems to be locking calls to getResolvedPathForZip(), which >> sets the pathToZip member if it's null. Is there a reason for using a >> lock for this instead of making pathToZip volatile like the offsets >> ArrayList is? >> > It looks like it was intended to support asynchronous close of the zip > file system, in which case it should be using fs.begin() and fs.end() > rather than its own lock (Rajendra - can you confirm this?). Hi Alan, I am alive :-) gone into snooze mode on jdk7 and working on different projects. Sorry for delay in responding the mail. The main intention of using lock is for each operation that accesses the zip file acquires the read lock, tests if the file system is closed and if so throws ClosedFileSystemException and at the end releases the lock. The close method acquires the write lock and will wait until all threads are finished access the zip file. Thanks Rajendra From Alan.Bateman at Sun.COM Fri Dec 4 05:17:08 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 04 Dec 2009 13:17:08 +0000 Subject: locking in zipfs demo In-Reply-To: <4B190481.2060709@Sun.com> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> <4B190481.2060709@Sun.com> Message-ID: <4B190BD4.7030202@sun.com> Rajendra.Gutupalli at Sun.com wrote: > Alan Bateman wrote: >> Joel Uckelman wrote: >>> I'm puzzled by the ReentrantReadWriteLock used in ZipFilePath in >>> various >>> place. It seems to be locking calls to getResolvedPathForZip(), which >>> sets the pathToZip member if it's null. Is there a reason for using a >>> lock for this instead of making pathToZip volatile like the offsets >>> ArrayList is? >>> >> It looks like it was intended to support asynchronous close of the >> zip file system, in which case it should be using fs.begin() and >> fs.end() rather than its own lock (Rajendra - can you confirm this?). > Hi Alan, > > I am alive :-) gone into snooze mode on jdk7 and working on > different projects. Sorry for delay in responding the mail. > > The main intention of using lock is for each operation that accesses > the zip file acquires the read lock, tests if the file system is > closed and if so throws ClosedFileSystemException > and at the end releases the lock. The close method acquires the write > lock and will wait until all threads are finished access the zip file. Thanks for confirming this - it looks like the readLock should be removed from ZipFilePath and its begin/end methods should be changed to invoke filesystem.begin() and filesystem.end(). That way operations on ZipFilePath will be work with the asynchronous close mechanism. Joel - do you want to take this one? -Alan. From hjohn at xs4all.nl Sat Dec 5 04:18:08 2009 From: hjohn at xs4all.nl (John Hendrikx) Date: Sat, 05 Dec 2009 13:18:08 +0100 Subject: WatchService questions Message-ID: <4B1A4F80.2070106@xs4all.nl> Hi, I'm making good use of the WatchService for a file manager type program, and I'm finding that I have some questions that I cannot glean from the documentation or examples. Let me give some quick background. I'm using the WatchService to monitor a directory. This directory sometimes is heavily modified (like deleting 7000 files, copying lots of files etc). I've been having some trouble during testing with high CPU use resulting from lots and lots of updates from the WatchService, which lead me to these questions. 1) How should OVERFLOW be handled? If it occurs, will it be the first event returned from pollEvents, and if so, can I safely reset the key after handling it? If not the first event, would it be wise to scan for an OVERFLOW and then just reset the key? 2) Is it possible to keep a directory sync'd using the WatchService or are there races/timing issues that may cause updates to be lost? Like for example, when I handle OVERFLOW (by re-reading the directory), what would be the correct way of making sure I'm not missing any updates? Currently I re-read the directory before resetting the key again, is this sufficient? In other words, can I rely on the WatchService to keep my directory up-to-date or should I periodically re-read the directory? 3) Is there any way to throttle the amount of updates? Currently I just delay for 1 second in the loop that keeps an eye on the WatchService -- without this, the handling of all the events (during Copy or Deletes) consumes so much CPU that some threads (Swing updates) are starved for CPU time. This may also be my own slow handling of the incoming events, but I did notice that ENTRY_MODIFY can be send a lot during copying (once for every few kB copied it seems, which makes for a fun live update display...). Are ENTRY_MODIFY events consolidated if I simply donot poll as often? Could I throttle only ENTRY_MODIFY? This is an outline of the code I currently use: for(;;) { Thread.sleep(1000); final WatchKey key = watchService.take(); try { EventQueue.invokeAndWait(new Runnable() { @Override public void run() { synchronized(listModel) { if(key.isValid()) { for(WatchEvent event : key.pollEvents()) { // Handle the events -- sometimes > dozens/second } } } } }); } catch(...) {} key.reset(); } I suspect my problem is that I'm handling most of the events on the Swing dispatch thread, and I'm underestimating how much time that is taking me (and how much events are getting generated). It functions as intended when copying large files, but when many small files are involved, the load is so great that Swing can't do it's regular updates any more. Thanks for any insights, John From uckelman at nomic.net Sat Dec 5 07:52:24 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Sat, 05 Dec 2009 16:52:24 +0100 Subject: locking in zipfs demo In-Reply-To: <4B190BD4.7030202@sun.com> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> <4B190481.2060709@Sun.com> <4B190BD4.7030202@sun.com> Message-ID: <20091205155225.24B10100D1@charybdis.ellipsis.cx> Thus spake Alan Bateman: > > Thanks for confirming this - it looks like the readLock should be > removed from ZipFilePath and its begin/end methods should be changed to > invoke filesystem.begin() and filesystem.end(). That way operations on > ZipFilePath will be work with the asynchronous close mechanism. > > Joel - do you want to take this one? > Done. Patch attached. -------------- next part -------------- # HG changeset patch # User Joel Uckelman # Date 1259871080 -3600 # Node ID c9509c24484b305e4564a730cc8356155bd8c2a9 # Parent 073e75b34c3d2fdb8c49d0ce259e9cedf85f1cc9 # Parent fa2afb1da3a0b800b4d95dd7c325231291d5e9b1 Merged from OpenJDK. diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipUtils.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipUtils.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipUtils.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipUtils.java @@ -200,10 +200,11 @@ filenameLen + extraFieldLen + commentLen; ZipFilePath entryPath = null; - entryPath = zipPath.resolve(new String(ze.filename)); + String fn = new String(ze.filename); + entryPath = zipPath.resolve(fn); ze.isArchiveFile = entryPath.isArchiveFile(); - ze.isDirectory = (entryPath.toString().endsWith("/") ? true : false); + ze.isDirectory = fn.endsWith("/"); ze.isRegularFile = !ze.isDirectory; if (isJar) { jentry = new JarEntryInfo(ze); diff --git a/test/demo/nio/ZipFileSystem/Sanity.java b/test/demo/nio/ZipFileSystem/Basic.java rename from test/demo/nio/ZipFileSystem/Sanity.java rename to test/demo/nio/ZipFileSystem/Basic.java --- a/test/demo/nio/ZipFileSystem/Sanity.java +++ b/test/demo/nio/ZipFileSystem/Basic.java @@ -29,10 +29,10 @@ import java.io.IOException; /** - * Sanity check zip provider by running a few simple tests. + * Basic test for zip provider */ -public class Sanity { +public class Basic { public static void main(String[] args) throws Exception { Path zipfile = Paths.get(args[0]); diff --git a/test/demo/nio/ZipFileSystem/PathOps.java b/test/demo/nio/ZipFileSystem/PathOps.java new file mode 100644 --- /dev/null +++ b/test/demo/nio/ZipFileSystem/PathOps.java @@ -0,0 +1,414 @@ +/* + * Copyright 2008-2009 Sun Microsystems, Inc. All Rights Reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, + * CA 95054 USA or visit www.sun.com if you need additional information or + * have any questions. + */ + +import java.nio.file.*; +import java.util.*; +import java.io.IOException; + +/** + * Tests path operations for zip provider. + */ + +public class PathOps { + + static final java.io.PrintStream out = System.out; + static FileSystem fs; + + private String input; + private Path path; + private Exception exc; + + private PathOps(String s) { + out.println(); + input = s; + try { + path = fs.getPath(s); + out.format("%s -> %s", s, path); + } catch (Exception x) { + exc = x; + out.format("%s -> %s", s, x); + } + out.println(); + } + + Path path() { + return path; + } + + void fail() { + throw new RuntimeException("PathOps failed"); + } + + void checkPath() { + if (path == null) { + throw new InternalError("path is null"); + } + } + + void check(Object result, String expected) { + out.format("\tExpected: %s\n", expected); + out.format("\tActual: %s\n", result); + if (result == null) { + if (expected == null) return; + } else { + // compare string representations + if (expected != null && result.toString().equals(expected.toString())) + return; + } + fail(); + } + + void check(Object result, boolean expected) { + check(result, Boolean.toString(expected)); + } + + PathOps root(String expected) { + out.println("check root"); + checkPath(); + check(path.getRoot(), expected); + return this; + } + + PathOps parent(String expected) { + out.println("check parent"); + checkPath(); + check(path.getParent(), expected); + return this; + } + + PathOps name(String expected) { + out.println("check name"); + checkPath(); + check(path.getName(), expected); + return this; + } + + PathOps element(int index, String expected) { + out.format("check element %d\n", index); + checkPath(); + check(path.getName(index), expected); + return this; + } + + PathOps subpath(int startIndex, int endIndex, String expected) { + out.format("test subpath(%d,%d)\n", startIndex, endIndex); + checkPath(); + check(path.subpath(startIndex, endIndex), expected); + return this; + } + + PathOps starts(String prefix) { + out.format("test startsWith with %s\n", prefix); + checkPath(); + Path s = fs.getPath(prefix); + check(path.startsWith(s), true); + return this; + } + + PathOps notStarts(String prefix) { + out.format("test not startsWith with %s\n", prefix); + checkPath(); + Path s = fs.getPath(prefix); + check(path.startsWith(s), false); + return this; + } + + PathOps ends(String suffix) { + out.format("test endsWith %s\n", suffix); + checkPath(); + Path s = fs.getPath(suffix); + check(path.endsWith(s), true); + return this; + } + + PathOps notEnds(String suffix) { + out.format("test not endsWith %s\n", suffix); + checkPath(); + Path s = fs.getPath(suffix); + check(path.endsWith(s), false); + return this; + } + + PathOps absolute() { + out.println("check path is absolute"); + checkPath(); + check(path.isAbsolute(), true); + return this; + } + + PathOps notAbsolute() { + out.println("check path is not absolute"); + checkPath(); + check(path.isAbsolute(), false); + return this; + } + + PathOps resolve(String other, String expected) { + out.format("test resolve %s\n", other); + checkPath(); + check(path.resolve(other), expected); + return this; + } + + PathOps relativize(String other, String expected) { + out.format("test relativize %s\n", other); + checkPath(); + Path that = fs.getPath(other); + check(path.relativize(that), expected); + return this; + } + + PathOps normalize(String expected) { + out.println("check normalized path"); + checkPath(); + check(path.normalize(), expected); + return this; + } + + PathOps string(String expected) { + out.println("check string representation"); + checkPath(); + check(path, expected); + return this; + } + + PathOps invalid() { + if (!(exc instanceof InvalidPathException)) { + out.println("InvalidPathException not thrown as expected"); + fail(); + } + return this; + } + + static PathOps test(String s) { + return new PathOps(s); + } + + // -- PathOpss -- + + static void header(String s) { + out.println(); + out.println(); + out.println("-- " + s + " --"); + } + + static void doPathOpTests() { + header("Path operations"); + + // all components + test("/a/b/c") + .root("/") + .parent("/a/b") + .name("c"); + + // root component only + test("/") + .root("/") + .parent(null) + .name(null); + + // no root component + test("a/b") + .root(null) + .parent("a") + .name("b"); + + // name component only + test("foo") + .root(null) + .parent(null) + .name("foo"); + + // startsWith + test("/") + .starts("/") + .notStarts("/foo"); + test("/foo") + .starts("/") + .starts("/foo") + .notStarts("/f"); + test("/foo/bar") + .starts("/") + .starts("/foo") + .starts("/foo/bar") + .notStarts("/f") + .notStarts("foo") + .notStarts("foo/bar"); + test("foo") + .starts("foo") + .notStarts("f"); + test("foo/bar") + .starts("foo") + .starts("foo/bar") + .notStarts("f") + .notStarts("/foo") + .notStarts("/foo/bar"); + + // endsWith + test("/") + .ends("/") + .notEnds("foo") + .notEnds("/foo"); + test("/foo") + .ends("foo") + .ends("/foo") + .notEnds("/"); + test("/foo/bar") + .ends("bar") + .ends("foo/bar") + .ends("/foo/bar") + .notEnds("/bar"); + test("foo") + .ends("foo"); + test("foo/bar") + .ends("bar") + .ends("foo/bar"); + + // elements + test("a/b/c") + .element(0,"a") + .element(1,"b") + .element(2,"c"); + + // isAbsolute + test("/") + .absolute(); + test("/tmp") + .absolute(); + test("tmp") + .notAbsolute(); + + // resolve + test("/tmp") + .resolve("foo", "/tmp/foo") + .resolve("/foo", "/foo"); + test("tmp") + .resolve("foo", "tmp/foo") + .resolve("/foo", "/foo"); + + // relativize + test("/a/b/c") + .relativize("/a/b/c", null) + .relativize("/a/b/c/d/e", "d/e") + .relativize("/a/x", "../../x"); + + // normalize + test("/") + .normalize("/"); + test("foo") + .normalize("foo"); + test("/foo") + .normalize("/foo"); + test(".") + .normalize(null); + test("..") + .normalize(".."); + test("/..") + .normalize("/"); + test("/../..") + .normalize("/"); + test("foo/.") + .normalize("foo"); + test("./foo") + .normalize("foo"); + test("foo/..") + .normalize(null); + test("../foo") + .normalize("../foo"); + test("../../foo") + .normalize("../../foo"); + test("foo/bar/..") + .normalize("foo"); + test("foo/bar/gus/../..") + .normalize("foo"); + test("/foo/bar/gus/../..") + .normalize("/foo"); + + // invalid + test("foo\u0000bar") + .invalid(); + test("\u0000foo") + .invalid(); + test("bar\u0000") + .invalid(); + test("//foo\u0000bar") + .invalid(); + test("//\u0000foo") + .invalid(); + test("//bar\u0000") + .invalid(); + + // normalization + test("//foo//bar") + .string("/foo/bar") + .root("/") + .parent("/foo") + .name("bar"); + } + + static void npes() { + header("NullPointerException"); + + Path path = fs.getPath("foo"); + + try { + path.resolve((String)null); + throw new RuntimeException("NullPointerException not thrown"); + } catch (NullPointerException npe) { + } + + try { + path.relativize(null); + throw new RuntimeException("NullPointerException not thrown"); + } catch (NullPointerException npe) { + } + + try { + path.compareTo(null); + throw new RuntimeException("NullPointerException not thrown"); + } catch (NullPointerException npe) { + } + + try { + path.startsWith(null); + throw new RuntimeException("NullPointerException not thrown"); + } catch (NullPointerException npe) { + } + + try { + path.endsWith(null); + throw new RuntimeException("NullPointerException not thrown"); + } catch (NullPointerException npe) { + } + + } + + public static void main(String[] args) throws IOException { + Path zipfile = Paths.get(args[0]); + Map env = new HashMap(); + fs = FileSystems.newFileSystem(zipfile, env, null); + + npes(); + doPathOpTests(); + } +} diff --git a/test/demo/nio/ZipFileSystem/sanity.sh b/test/demo/nio/ZipFileSystem/basic.sh rename from test/demo/nio/ZipFileSystem/sanity.sh rename to test/demo/nio/ZipFileSystem/basic.sh --- a/test/demo/nio/ZipFileSystem/sanity.sh +++ b/test/demo/nio/ZipFileSystem/basic.sh @@ -23,9 +23,9 @@ # @test # @bug 4313887 -# @summary Sanity check ZipFileSystem demo -# @build Sanity -# @run shell sanity.sh +# @summary Test ZipFileSystem demo +# @build Basic PathOps +# @run shell basic.sh if [ -z "${TESTJAVA}" ]; then echo "Test must be run with jtreg" @@ -59,7 +59,8 @@ # Run the tests -go Sanity "${ZIPFS}" +go Basic "${ZIPFS}" +go PathOps "${ZIPFS}" # # Results From uckelman at nomic.net Sat Dec 5 08:02:01 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Sat, 05 Dec 2009 17:02:01 +0100 Subject: locking in zipfs demo In-Reply-To: <20091205155225.24B10100D1@charybdis.ellipsis.cx> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> <4B190481.2060709@Sun.com> <4B190BD4.7030202@sun.com> <20091205155225.24B10100D1@charybdis.ellipsis.cx> Message-ID: <20091205160201.C1B2A100D1@charybdis.ellipsis.cx> Thus spake Joel Uckelman: > > Done. Patch attached. > Ack, I exported the wrong changeset. Here's the patch I meant to send: -------------- next part -------------- # HG changeset patch # User Joel Uckelman # Date 1260027861 -3600 # Node ID 4cf3aee1209b5c2ef886ee7f66a96ccb352af98f # Parent c9509c24484b305e4564a730cc8356155bd8c2a9 Switched to use FS read lock (to prevent FS closure), not Path-local read lock. diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java @@ -45,8 +45,6 @@ import java.net.URI; import java.nio.channels.FileChannel; import java.nio.file.attribute.Attributes; -import java.util.concurrent.locks.ReentrantReadWriteLock; -import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock; import java.util.regex.Matcher; import java.util.regex.Pattern; import java.util.zip.ZipEntry; @@ -71,7 +69,6 @@ // resolved path for locating zip inside zip file // resloved path does not contain ./ and .. components private final byte[] pathForZip; - private final ReadLock readLock = new ReentrantReadWriteLock().readLock(); private ZipFilePath pathToZip; private final byte[] pathForPrint; @@ -143,7 +140,7 @@ */ public boolean isDirectory() { try { - begin(); + fileSystem.begin(); try { ZipFilePath resolved = getResolvedPathForZip(); return Attributes.readBasicFileAttributes(resolved, LinkOption.NOFOLLOW_LINKS).isDirectory(); @@ -151,7 +148,7 @@ return false; } } finally { - end(); + fileSystem.end(); } } @@ -165,17 +162,6 @@ return index; } - final void begin() { - readLock.lock(); - if (!fileSystem.isOpen()) { - throw new ClosedFileSystemException(); - } - } - - final void end() { - readLock.unlock(); - } - static int nextNonSeparator(byte[] path, int index) { int length = path.length; @@ -822,7 +808,7 @@ } } try { - begin(); + fileSystem.begin(); ZipFilePath realPath = getResolvedPathForZip(); if (realPath.getNameCount() == 0) { throw new IOException("entry missing in the path"); @@ -839,7 +825,7 @@ return new ZipFilePathInputStream(zfile.getInputStream(entry)); } } finally { - end(); + fileSystem.end(); } } @@ -849,10 +835,10 @@ Filter filter) throws IOException { try { - begin(); + fileSystem.begin(); return new ZipFileStream(getResolvedPathForZip(), filter); } finally { - end(); + fileSystem.end(); } } private static final DirectoryStream.Filter acceptAllFilter = @@ -967,14 +953,14 @@ @Override public FileStore getFileStore() throws IOException { try { - begin(); + fileSystem.begin(); if (isAbsolute()) { return ZipFileStore.create(getRoot()); } else { return ZipFileStore.create(getResolvedPathForZip().getRoot()); } } finally { - end(); + fileSystem.end(); } } @@ -1076,7 +1062,7 @@ throw new IllegalArgumentException("not opened for Read"); //this is never thrown } try { - begin(); + fileSystem.begin(); ZipFilePath realPath = getResolvedPathForZip(); if (realPath.getNameCount() == 0) { throw new IOException("entry Not Found"); @@ -1098,7 +1084,7 @@ ); } } finally { - end(); + fileSystem.end(); } } @@ -1150,7 +1136,7 @@ } try { - begin(); + fileSystem.begin(); ZipFilePath resolvedZipPath = getResolvedPathForZip(); int nameCount = resolvedZipPath.getNameCount(); if (nameCount == 0) { @@ -1167,7 +1153,7 @@ } } } finally { - end(); + fileSystem.end(); } } From Alan.Bateman at Sun.COM Sat Dec 5 12:48:53 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 05 Dec 2009 20:48:53 +0000 Subject: WatchService questions In-Reply-To: <4B1A4F80.2070106@xs4all.nl> References: <4B1A4F80.2070106@xs4all.nl> Message-ID: <4B1AC735.50404@sun.com> John Hendrikx wrote: > Hi, I'm making good use of the WatchService for a file manager type > program, and I'm finding that I have some questions that I cannot > glean from the documentation or examples. It's good for us to get feedback from folks that are using this API in anger. > > Let me give some quick background. I'm using the WatchService to > monitor a directory. This directory sometimes is heavily modified > (like deleting 7000 files, copying lots of files etc). I've been > having some trouble during testing with high CPU use resulting from > lots and lots of updates from the WatchService, which lead me to these > questions. > > 1) How should OVERFLOW be handled? If it occurs, will it be the first > event returned from pollEvents, and if so, can I safely reset the key > after handling it? If not the first event, would it be wise to scan > for an OVERFLOW and then just reset the key? Are you seeing OVERFLOW events? Our implementations are reasonably efficient at draining the events from the kernel so I would expect it to be very rare that the overflow is caused by reaching the kernel limits. There is a second limit in the watch service that is another source of OVERFLOW events. If the application doesn't retrieve the events in a timely manner then events will accumulate. If the same files are modified many times then it's not a problem because it just increments the modification count. Create/delete, on the other hand, are distinct events and so will queue up. Our implementation limits the number of distinct events pending per directory to 512. That limit was chosen arbitrarily and unfortunately isn't yet configurable. I can't say if this is what you are running into but if you are seeing a lot of OVERFLOW events then I'll bet that is the issue. Looking at the code snippet below it appears that the thread that waits for keys to be signalled also waits for the AWT event thread to retrieve and process the events. Could this be changed to retrieve the events and invoked asynchronously on the AWT event thread (with invokeLater)? As regards dealing with the OVERFLOW event then it is simply an indication that events have potentially been lost so you should refresh your view of the directory and simply continue processing events. It may not be the first event (and appear anywhere in the list). They key remains valid and so can be reset. > > 2) Is it possible to keep a directory sync'd using the WatchService or > are there races/timing issues that may cause updates to be lost? Like > for example, when I handle OVERFLOW (by re-reading the directory), > what would be the correct way of making sure I'm not missing any > updates? Currently I re-read the directory before resetting the key > again, is this sufficient? In other words, can I rely on the > WatchService to keep my directory up-to-date or should I periodically > re-read the directory? Yes, the watch service will keep your view of directory in sync and I can't think of any issues or bugs that would cause it not to be in sync. In other words, it shouldn't be necessary to periodically iterate over the directory to refresh your view. There are of course timing issue where you view is temporarily out of sync but that shouldn't be a problem. When you receive an OVERFLOW then you refresh your view and process any subsequent events as normal. I should say that after you refresh you may process a number of pending events that you will likely ignore -- for example, suppose a ENTRY_CREATE event is queued after an OVERFLOW event. When you refresh your view will see see the new file and so the subsequent ENTRY_CREATE event will not update your view. Does that make sense? > > 3) Is there any way to throttle the amount of updates? Currently I > just delay for 1 second in the loop that keeps an eye on the > WatchService -- without this, the handling of all the events (during > Copy or Deletes) consumes so much CPU that some threads (Swing > updates) are starved for CPU time. This may also be my own slow > handling of the incoming events, but I did notice that ENTRY_MODIFY > can be send a lot during copying (once for every few kB copied it > seems, which makes for a fun live update display...). Are > ENTRY_MODIFY events consolidated if I simply donot poll as often? > Could I throttle only ENTRY_MODIFY? I'm interested to know which threads are busy. Also, I'm interested to know if you've looked at the -verbose:gc output in case there is something else going on. It's platform dependent but for the platforms that I think you are on then there is a background thread draining the kernel buffers but it should be barely noticeable (even under load). : > > > I suspect my problem is that I'm handling most of the events on the > Swing dispatch thread, and I'm underestimating how much time that is > taking me (and how much events are getting generated). It functions > as intended when copying large files, but when many small files are > involved, the load is so great that Swing can't do it's regular > updates any more. Kernel will generate a lot of events during file copying but that case will often just cause the count for the last modification event to be incremented. Out of curiosity, do you actually need modification events? I probably don't have the full context here but if you just want to maintain a list of the files in a directory then the ENRTY_CREATE and ENTRY_DELETE events should be sufficient. -Alan. From Alan.Bateman at Sun.COM Sat Dec 5 12:49:57 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 05 Dec 2009 20:49:57 +0000 Subject: locking in zipfs demo In-Reply-To: <20091205160201.C1B2A100D1@charybdis.ellipsis.cx> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> <4B190481.2060709@Sun.com> <4B190BD4.7030202@sun.com> <20091205155225.24B10100D1@charybdis.ellipsis.cx> <20091205160201.C1B2A100D1@charybdis.ellipsis.cx> Message-ID: <4B1AC775.4000707@sun.com> Joel Uckelman wrote: > Thus spake Joel Uckelman: > >> Done. Patch attached. >> >> > > Ack, I exported the wrong changeset. Here's the patch I meant to send: > > Joel - another approach to consider would be simply change ZipFilePath.{begin,end} to invoke fileSystem.{begin,end}. It would avoid changing each usage (I don't have a strong opinion either way). -Alan From uckelman at nomic.net Sat Dec 5 13:42:17 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Sat, 05 Dec 2009 22:42:17 +0100 Subject: locking in zipfs demo In-Reply-To: <4B1AC775.4000707@sun.com> References: <20091202232759.F4229100D1@charybdis.ellipsis.cx> <4B177BF4.5000604@sun.com> <4B190481.2060709@Sun.com> <4B190BD4.7030202@sun.com> <20091205155225.24B10100D1@charybdis.ellipsis.cx> <20091205160201.C1B2A100D1@charybdis.ellipsis.cx> <4B1AC775.4000707@sun.com> Message-ID: <20091205214218.2ACCF100D1@charybdis.ellipsis.cx> Thus spake Alan Bateman: > > Joel - another approach to consider would be simply change > ZipFilePath.{begin,end} to invoke fileSystem.{begin,end}. It would avoid > changing each usage (I don't have a strong opinion either way). > I think calling the lock via an FS method makes it clearer that it's an FS-level lock. Calling it via a Path method was the thing which made me wonder whether it was correct in the first place. -- J. From hjohn at xs4all.nl Sun Dec 6 03:53:12 2009 From: hjohn at xs4all.nl (John Hendrikx) Date: Sun, 06 Dec 2009 12:53:12 +0100 Subject: WatchService questions In-Reply-To: <4B1AC735.50404@sun.com> References: <4B1A4F80.2070106@xs4all.nl> <4B1AC735.50404@sun.com> Message-ID: <4B1B9B28.9090007@xs4all.nl> >> >> Let me give some quick background. I'm using the WatchService to >> monitor a directory. This directory sometimes is heavily modified >> (like deleting 7000 files, copying lots of files etc). I've been >> having some trouble during testing with high CPU use resulting from >> lots and lots of updates from the WatchService, which lead me to >> these questions. >> >> 1) How should OVERFLOW be handled? If it occurs, will it be the >> first event returned from pollEvents, and if so, can I safely reset >> the key after handling it? If not the first event, would it be wise >> to scan for an OVERFLOW and then just reset the key? > Are you seeing OVERFLOW events? Our implementations are reasonably > efficient at draining the events from the kernel so I would expect it > to be very rare that the overflow is caused by reaching the kernel > limits. There is a second limit in the watch service that is another > source of OVERFLOW events. If the application doesn't retrieve the > events in a timely manner then events will accumulate. If the same > files are modified many times then it's not a problem because it just > increments the modification count. Create/delete, on the other hand, > are distinct events and so will queue up. Our implementation limits > the number of distinct events pending per directory to 512. That limit > was chosen arbitrarily and unfortunately isn't yet configurable. I > can't say if this is what you are running into but if you are seeing a > lot of OVERFLOW events then I'll bet that is the issue. Looking at the > code snippet below it appears that the thread that waits for keys to > be signalled also waits for the AWT event thread to retrieve and > process the events. Could this be changed to retrieve the events and > invoked asynchronously on the AWT event thread (with invokeLater)? Yes, I'm seeing OVERFLOW events. I'm not processing them fast enough as I'm updating the UI, which admittedly isn't highly optimized. Buffering them is an option, although even such a buffer could overflow. It would however give me the opportunity to deal with a huge burst of events in my own manner: for example, clear the buffer if OVERFLOW is received and then insert the OVERFLOW event, or even clear the buffer when the events exceed a certain maximum and insert my own OVERFLOW. Anyway, I'm pretty sure the OVERFLOW events are caused by slow retrieval as the UI processing involves updating a JTable, so I donot think I'm hitting the Kernel limit here. The throttling of events I'm doing is also playing a big part in this. >> 2) Is it possible to keep a directory sync'd using the WatchService >> or are there races/timing issues that may cause updates to be lost? >> Like for example, when I handle OVERFLOW (by re-reading the >> directory), what would be the correct way of making sure I'm not >> missing any updates? Currently I re-read the directory before >> resetting the key again, is this sufficient? In other words, can I >> rely on the WatchService to keep my directory up-to-date or should I >> periodically re-read the directory? > > Yes, the watch service will keep your view of directory in sync and I > can't think of any issues or bugs that would cause it not to be in > sync. In other words, it shouldn't be necessary to periodically > iterate over the directory to refresh your view. There are of course > timing issue where you view is temporarily out of sync but that > shouldn't be a problem. When you receive an OVERFLOW then you refresh > your view and process any subsequent events as normal. I should say > that after you refresh you may process a number of pending events that > you will likely ignore -- for example, suppose a ENTRY_CREATE event is > queued after an OVERFLOW event. When you refresh your view will see > see the new file and so the subsequent ENTRY_CREATE event will not > update your view. Does that make sense? Yes, I think I understand -- that's also what I've been seeing and I had to adapt my event handling to take into account events that were already "handled" as part of re-reading the directory -- so it is also important to first initialize a WatchKey, then read the directory, not the other way around :) Now that I think about it, this makes a lot of sense and allows implementations to keep a directory sync'd pretty easily. What I hadn't realized is that after you receive an OVERFLOW that the subsequent events will form a cohesive stream of events again which can be used to update a directory after re-reading it. It does make sense to me though to "scan" for the overflow so to speak -- either that or I would probably expect an implementation of WatchService to clear all pending events before adding the OVERFLOW. Not only are the pending events useless if an overflow occurs, but clearing them would make room for subsequent events again (if not, you'd be in a permanent state of overflow...?) >> 3) Is there any way to throttle the amount of updates? Currently I >> just delay for 1 second in the loop that keeps an eye on the >> WatchService -- without this, the handling of all the events (during >> Copy or Deletes) consumes so much CPU that some threads (Swing >> updates) are starved for CPU time. This may also be my own slow >> handling of the incoming events, but I did notice that ENTRY_MODIFY >> can be send a lot during copying (once for every few kB copied it >> seems, which makes for a fun live update display...). Are >> ENTRY_MODIFY events consolidated if I simply donot poll as often? >> Could I throttle only ENTRY_MODIFY? > I'm interested to know which threads are busy. Also, I'm interested to > know if you've looked at the -verbose:gc output in case there is > something else going on. It's platform dependent but for the platforms > that I think you are on then there is a background thread draining the > kernel buffers but it should be barely noticeable (even under load). My original assessment was wrong. The Swing Event thread was not starved for CPU, but instead was busy handling the WatchService events in the invokeAndWait part. This made it seem that the Event thread was starved for CPU (as other parts of the UI weren't updating anymore) while it really was consuming the CPU all by itself. In effect, in the time it took me to handle 60 WatchService events, 513 (512 + Overflow?) more had accumulated, resulting in second long delays for the UI to become responsive again. > Kernel will generate a lot of events during file copying but that case > will often just cause the count for the last modification event to be > incremented. Out of curiosity, do you actually need modification > events? I probably don't have the full context here but if you just > want to maintain a list of the files in a directory then the > ENRTY_CREATE and ENTRY_DELETE events should be sufficient. I do need them, as I need to know changes in file size as well. Most of the problems I've been having are related to underestimating the amount of events that could be generated. Now that I have a clear picture of how to deal with overflows and where the bottleneck is I should be able to solve the problems I've been having. Thanks for your help! --John From Alan.Bateman at Sun.COM Sun Dec 6 09:44:27 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 06 Dec 2009 17:44:27 +0000 Subject: WatchService questions In-Reply-To: <4B1B9B28.9090007@xs4all.nl> References: <4B1A4F80.2070106@xs4all.nl> <4B1AC735.50404@sun.com> <4B1B9B28.9090007@xs4all.nl> Message-ID: <4B1BED7B.40103@sun.com> John Hendrikx wrote: > : > > What I hadn't realized is that after you receive an OVERFLOW that the > subsequent events will form a cohesive stream of events again which > can be used to update a directory after re-reading it. It does make > sense to me though to "scan" for the overflow so to speak -- either > that or I would probably expect an implementation of WatchService to > clear all pending events before adding the OVERFLOW. Not only are the > pending events useless if an overflow occurs, but clearing them would > make room for subsequent events again (if not, you'd be in a permanent > state of overflow...?) This is a good point as the watch service could drop all pending events when the limit is reached. Furthermore it can drop all subsequent events while the head is an OVERFLOW event. I'll create a bug to track this - thanks! > My original assessment was wrong. The Swing Event thread was not > starved for CPU, but instead was busy handling the WatchService events > in the invokeAndWait part. This made it seem that the Event thread > was starved for CPU (as other parts of the UI weren't updating > anymore) while it really was consuming the CPU all by itself. > > In effect, in the time it took me to handle 60 WatchService events, > 513 (512 + Overflow?) more had accumulated, resulting in second long > delays for the UI to become responsive again. Thanks for the update. I did a quick test that watches a directory and retrieves all events in a timely manner. Another program creates and deletes 10000 files in the watched directory which I think is what you are doing. The test was on Windows (as I think this is where your UI is at, right?). As expected the watcher didn't consume too many CPU cycles and is barely noticeable. Also, the list of events typically only contain a single event as it can easily keep up with programs creating lots of files (which is a very slow in comparison). I changed the test to sleep periodically, thus delaying the retrieval of events. In that case, 512 events are queued very quickly, and subsequent events are dropped. Again, the CPU usage is barely noticeable so this concurs with your observation that the time is spent on processing the events. Anyway, let us know how your solution goes - your war story may be useful to others. -Alan. From Alan.Bateman at Sun.COM Sun Dec 6 09:47:44 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 06 Dec 2009 17:47:44 +0000 Subject: Selector doesn't implement Closeable In-Reply-To: <4B18121B.2080209@sun.com> References: <4B08347B.6000502@univ-mlv.fr> <4B094AF5.5030209@sun.com> <4B18121B.2080209@sun.com> Message-ID: <4B1BEE40.7090201@sun.com> I didn't hear back from R?mi on this but I might as well push this into b78 and be done with it. Could I get a reviewer? Thanks, Alan. Alan Bateman wrote: > : > > diff -r d5a1c012921d src/share/classes/java/nio/channels/Selector.java > --- a/src/share/classes/java/nio/channels/Selector.java Sun Nov 29 > 15:24:32 2009 -0800 > +++ b/src/share/classes/java/nio/channels/Selector.java Thu Dec 03 > 19:29:04 2009 +0000 > @@ -25,6 +25,7 @@ > > package java.nio.channels; > > +import java.io.Closeable; > import java.io.IOException; > import java.nio.channels.spi.SelectorProvider; > import java.util.Set; > @@ -202,7 +203,7 @@ import java.util.Set; > * @see SelectionKey > */ > > -public abstract class Selector { > +public abstract class Selector implements Closeable { > > /** > * Initializes a new instance of this class. From uckelman at nomic.net Sun Dec 6 11:42:58 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Sun, 06 Dec 2009 20:42:58 +0100 Subject: more zipfs patches Message-ID: <20091206194259.5E975100D4@charybdis.ellipsis.cx> Two more zipfs patches: * Added ZipFileAttributeView.getAttribute(). * Renamed ZipFileAttributes.getExternalAttrs() to externalAttrs() for consistency. * Bug: Must clear milliseconds before using Calendar for converting DOS time. * Bug: isSameFile() now handles nonexistent files properly -------------- next part -------------- # HG changeset patch # User Joel Uckelman # Date 1260105113 -3600 # Node ID b713729d52c0afb8181ee21206bcc6298996438a # Parent 4cf3aee1209b5c2ef886ee7f66a96ccb352af98f * Added ZipFileAttributeView.getAttribute(). * Renamed ZipFileAttributes.getExternalAttrs() to externalAttrs() for consistency. * Must clear milliseconds before using Calendar for converting DOS time. diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributeView.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributeView.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributeView.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributeView.java @@ -50,4 +50,47 @@ throws IOException { return new ZipFileAttributes(file); } + + @Override + public Object getAttribute(String attribute) throws IOException { + final ZipFileAttributes zfa = readAttributes(); + + if (attribute.equals("comment")) { + return zfa.comment(); + } + + if (attribute.equals("compressedSize")) { + return zfa.compressedSize(); + } + + if (attribute.equals("crc")) { + return zfa.crc(); + } + + if (attribute.equals("extra")) { + return zfa.extra(); + } + + if (attribute.equals("method")) { + return zfa.method(); + } + + if (attribute.equals("name")) { + return zfa.name(); + } + + if (attribute.equals("isArchiveFile")) { + return zfa.isArchiveFile(); + } + + if (attribute.equals("versionMadeBy")) { + return zfa.versionMadeBy(); + } + + if (attribute.equals("extAttrs")) { + return zfa.externalAttrs(); + } + + return null; + } } diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributes.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributes.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributes.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileAttributes.java @@ -37,9 +37,25 @@ public class ZipFileAttributes extends ZipFileBasicAttributes { /** Creates a new instance of ZipFileAttributes */ - private String[] version = {"FAT file system (DOS, OS/2, NT)", "Amiga", "VMS (VAX or Alpha AXP)", "Unix", "VM/CMS", "Atari", "HPFS file system (OS/2, NT 3.x)", - "Macintosh", "Z-System", "CP/M", "TOPS-20", "NTFS file system (NT)", "SMS/QDOS", "Acorn RISC OS", "VFAT file system (Win95, NT)", - "MVS", "BeOS (BeBox or PowerMac)", "Tandem" + private String[] version = { + "FAT file system (DOS, OS/2, NT)", + "Amiga", + "VMS (VAX or Alpha AXP)", + "UNIX", + "VM/CMS", + "Atari", + "HPFS file system (OS/2, NT 3.x)", + "Macintosh", + "Z-System", + "CP/M", + "TOPS-20", + "NTFS file system (NT)", + "SMS/QDOS", + "Acorn RISC OS", + "VFAT file system (Win95, NT)", + "MVS", + "BeOS (BeBox or PowerMac)", + "Tandem" }; public ZipFileAttributes(FileRef file) @@ -85,7 +101,7 @@ } } - public int getExternalAttrs() { + public int externalAttrs() { return ze.extAttrs; } } diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileBasicAttributes.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileBasicAttributes.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileBasicAttributes.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFileBasicAttributes.java @@ -78,13 +78,13 @@ @Override public FileTime lastModifiedTime() { - long time = ze.lastModifiedTime; - Calendar cal = dosTimeToJavaTime(time); + final Calendar cal = dosTimeToJavaTime(ze.lastModifiedTime); return FileTime.fromMillis(cal.getTimeInMillis()); } private Calendar dosTimeToJavaTime(long time) { - Calendar cal = Calendar.getInstance(); + final Calendar cal = Calendar.getInstance(); + cal.clear(); // to set the milliseconds 0 cal.set((int) (((time >> 25) & 0x7f) + 1980), (int) (((time >> 21) & 0x0f) - 1), (int) ((time >> 16) & 0x1f), -------------- next part -------------- # HG changeset patch # User Joel Uckelman # Date 1260114056 -3600 # Node ID 36841187ed0717208ca2862bd60d4519d18a9b17 # Parent b713729d52c0afb8181ee21206bcc6298996438a isSameFile() now handles nonexistent files properly diff --git a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java --- a/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java +++ b/src/share/demo/nio/ZipFileSystem/com/sun/nio/zipfs/ZipFilePath.java @@ -966,28 +966,28 @@ @Override public boolean isSameFile(Path other) throws IOException { + if (this.equals(other)) { + return true; + } - if ((other != null) && (other instanceof ZipFilePath)) { + if (other == null || (!(other instanceof ZipFilePath))) { + return false; + } + + final ZipFilePath other1 = (ZipFilePath) other; - // check both file systems are same. + // check whether we have a common file system + if (!this.getFileSystem().getZipFileSystemFile().equals( + other1.getFileSystem().getZipFileSystemFile())) { + return false; + } - ZipFilePath other1 = (ZipFilePath) other; - String fileSystem1 = this.getFileSystem().getZipFileSystemFile(); - String fileSystem2 = other1.getFileSystem().getZipFileSystemFile(); - boolean isSameFileSystem = fileSystem1.equals(fileSystem2); - if (!isSameFileSystem) { - return false; - } + // check whether both (or neither) exist + if (this.exists() != other1.exists()) { + return false; + } - // if file systems are same then do they exist - // finally compare the paths - // compare the real paths - ZipFilePath thisZip = this.toRealPath(false); - ZipFilePath otherZip = other1.toRealPath(false); - return (thisZip.startsWith(otherZip) && thisZip.endsWith(otherZip)); - } - return false; - + return this.toAbsolutePath().equals(other1.toAbsolutePath()); } public WatchKey register( From Christopher.Hegarty at Sun.COM Mon Dec 7 03:05:11 2009 From: Christopher.Hegarty at Sun.COM (Christopher Hegarty - Sun Microsystems Ireland) Date: Mon, 07 Dec 2009 11:05:11 +0000 Subject: Selector doesn't implement Closeable In-Reply-To: <4B1BEE40.7090201@sun.com> References: <4B08347B.6000502@univ-mlv.fr> <4B094AF5.5030209@sun.com> <4B18121B.2080209@sun.com> <4B1BEE40.7090201@sun.com> Message-ID: <4B1CE167.3020700@Sun.COM> The change looks fine. Thanks Alan, -Chris. On 06/12/2009 17:47, Alan Bateman wrote: > > I didn't hear back from R?mi on this but I might as well push this into > b78 and be done with it. Could I get a reviewer? > > Thanks, > Alan. > > Alan Bateman wrote: >> : >> >> diff -r d5a1c012921d src/share/classes/java/nio/channels/Selector.java >> --- a/src/share/classes/java/nio/channels/Selector.java Sun Nov 29 >> 15:24:32 2009 -0800 >> +++ b/src/share/classes/java/nio/channels/Selector.java Thu Dec 03 >> 19:29:04 2009 +0000 >> @@ -25,6 +25,7 @@ >> >> package java.nio.channels; >> >> +import java.io.Closeable; >> import java.io.IOException; >> import java.nio.channels.spi.SelectorProvider; >> import java.util.Set; >> @@ -202,7 +203,7 @@ import java.util.Set; >> * @see SelectionKey >> */ >> >> -public abstract class Selector { >> +public abstract class Selector implements Closeable { >> >> /** >> * Initializes a new instance of this class. > From forax at univ-mlv.fr Mon Dec 7 03:49:39 2009 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Mon, 07 Dec 2009 12:49:39 +0100 Subject: Selector doesn't implement Closeable In-Reply-To: <4B1CE167.3020700@Sun.COM> References: <4B08347B.6000502@univ-mlv.fr> <4B094AF5.5030209@sun.com> <4B18121B.2080209@sun.com> <4B1BEE40.7090201@sun.com> <4B1CE167.3020700@Sun.COM> Message-ID: <4B1CEBD3.1090606@univ-mlv.fr> Sorry for my late answer, I'm not realiable these days. Change looks fine. R?mi Christopher Hegarty - Sun Microsystems Ireland a ?crit : > The change looks fine. Thanks Alan, > > -Chris. > > On 06/12/2009 17:47, Alan Bateman wrote: >> >> I didn't hear back from R?mi on this but I might as well push this >> into b78 and be done with it. Could I get a reviewer? >> >> Thanks, >> Alan. >> >> Alan Bateman wrote: >>> : >>> >>> diff -r d5a1c012921d src/share/classes/java/nio/channels/Selector.java >>> --- a/src/share/classes/java/nio/channels/Selector.java Sun Nov 29 >>> 15:24:32 2009 -0800 >>> +++ b/src/share/classes/java/nio/channels/Selector.java Thu Dec 03 >>> 19:29:04 2009 +0000 >>> @@ -25,6 +25,7 @@ >>> >>> package java.nio.channels; >>> >>> +import java.io.Closeable; >>> import java.io.IOException; >>> import java.nio.channels.spi.SelectorProvider; >>> import java.util.Set; >>> @@ -202,7 +203,7 @@ import java.util.Set; >>> * @see SelectionKey >>> */ >>> >>> -public abstract class Selector { >>> +public abstract class Selector implements Closeable { >>> >>> /** >>> * Initializes a new instance of this class. >> From Alan.Bateman at Sun.COM Tue Dec 8 09:05:49 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 08 Dec 2009 17:05:49 +0000 Subject: DoubleBuffer.compareTo is not anti-symmetric In-Reply-To: <1ccfd1c10911251903w716923c7xd5cae6e9ec0dba18@mail.gmail.com> References: <1ccfd1c10911201119t6c23c12u8fd890e8d79d73f1@mail.gmail.com> <4B07070D.3070404@sun.com> <1ccfd1c10911201826y2ff718eftf612e9024831a87e@mail.gmail.com> <4B095BA7.6010604@sun.com> <1ccfd1c10911231527i7a867ebbwc82820c9001d0215@mail.gmail.com> <4B0BC2E4.5060409@sun.com> <1ccfd1c10911241823y77681654hbc4ee989a23d7ff4@mail.gmail.com> <4B0D41D6.8050605@sun.com> <1ccfd1c10911251903w716923c7xd5cae6e9ec0dba18@mail.gmail.com> Message-ID: <4B1E876D.8070602@sun.com> Martin, Just to get closure on this one - I'm happy with the latest webrev so consider it reviewed. -Alan. From Thomas.Salter at unisys.com Fri Dec 11 08:17:11 2009 From: Thomas.Salter at unisys.com (Salter, Thomas A) Date: Fri, 11 Dec 2009 10:17:11 -0600 Subject: nio channels with IPv6 on Windows XP Message-ID: It seems that AsynchronousServerSocketChannel does not work with IPv6 sockets on Windows XP (the combination that invokes the TwoStack implementations). Is this a known restriction or is it intended to work? I can certainly understand why you might not want to support it. I tested on b77. My test runs on Windows 7 but fails on Windows XP. It fails on the bind because an IPv6 address is passed to AF_INET socket. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20091211/6e6148d0/attachment.html From Christopher.Hegarty at Sun.COM Fri Dec 11 08:36:17 2009 From: Christopher.Hegarty at Sun.COM (Christopher Hegarty - Sun Microsystems Ireland) Date: Fri, 11 Dec 2009 16:36:17 +0000 Subject: nio channels with IPv6 on Windows XP In-Reply-To: References: Message-ID: <4B227501.3010604@Sun.COM> Thomas, Windows IPv6 support was added to the NIO channels in JDK7 b36, and was only supported on Windows platforms that have a dual TCP/IP stack, i.e. Windows Vista (and greater). So you correct, it is a known limitation and by design. See : http://bugs.sun.com/view_bug.do?bug_id=6230761 -Chris. On 11/12/2009 16:17, Salter, Thomas A wrote: > It seems that AsynchronousServerSocketChannel does not work with IPv6 sockets on Windows XP (the combination that invokes the TwoStack implementations). Is this a known restriction or is it intended to work? I can certainly understand why you might not want to support it. > > I tested on b77. My test runs on Windows 7 but fails on Windows XP. It fails on the bind because an IPv6 address is passed to AF_INET socket. > > > From Alan.Bateman at Sun.COM Mon Dec 14 00:47:37 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 14 Dec 2009 08:47:37 +0000 Subject: nio channels with IPv6 on Windows XP In-Reply-To: References: Message-ID: <4B25FBA9.6060404@sun.com> Salter, Thomas A wrote: > It seems that AsynchronousServerSocketChannel does not work with IPv6 > sockets on Windows XP (the combination that invokes the TwoStack > implementations). Is this a known restriction or is it intended to > work? I can certainly understand why you might not want to support it. > > I tested on b77. My test runs on Windows 7 but fails on Windows XP. > It fails on the bind because an IPv6 address is passed to AF_INET socket. > As Chris said, you need Vista or newer because that's the first release of Windows with the new stack that supports dual-stack sockets (ie: interoperability with IPv4 and support for IPv4-mapped IPv6 addresses). The IPv6 stack that came in one of the Windows XP service packs was a separate stack and lacks IPv4 interoperability. -Alan. From mthornton at optrak.co.uk Mon Dec 14 00:53:02 2009 From: mthornton at optrak.co.uk (Mark Thornton) Date: Mon, 14 Dec 2009 08:53:02 +0000 Subject: nio channels with IPv6 on Windows XP In-Reply-To: <4B25FBA9.6060404@sun.com> References: <4B25FBA9.6060404@sun.com> Message-ID: <4B25FCEE.3060306@optrak.co.uk> Alan Bateman wrote: > Salter, Thomas A wrote: >> It seems that AsynchronousServerSocketChannel does not work with IPv6 >> sockets on Windows XP (the combination that invokes the TwoStack >> implementations). Is this a known restriction or is it intended to >> work? I can certainly understand why you might not want to support it. >> >> I tested on b77. My test runs on Windows 7 but fails on Windows XP. >> It fails on the bind because an IPv6 address is passed to AF_INET >> socket. >> > As Chris said, you need Vista or newer because that's the first > release of Windows with the new stack that supports dual-stack sockets > (ie: interoperability with IPv4 and support for IPv4-mapped IPv6 > addresses). The IPv6 stack that came in one of the Windows XP service > packs was a separate stack and lacks IPv4 interoperability. > > -Alan. Unfortunately you then find that InterfaceAddress.getBroadcast() and InterfaceAddress.getNetworkPrefixLength() returns nonsense for IPv4 addresses. The work around is to disable the IPv6 support ... http://bugs.sun.com/view_bug.do?bug_id=6707289 Mark Thornton From Alan.Bateman at Sun.COM Mon Dec 14 01:15:03 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 14 Dec 2009 09:15:03 +0000 Subject: nio channels with IPv6 on Windows XP In-Reply-To: <4B25FCEE.3060306@optrak.co.uk> References: <4B25FBA9.6060404@sun.com> <4B25FCEE.3060306@optrak.co.uk> Message-ID: <4B260217.8080601@sun.com> Mark Thornton wrote: > : > Unfortunately you then find that InterfaceAddress.getBroadcast() and > InterfaceAddress.getNetworkPrefixLength() returns nonsense for IPv4 > addresses. The work around is to disable the IPv6 support ... > > http://bugs.sun.com/view_bug.do?bug_id=6707289 I don't think this one is related but you should bring it up on net-dev as it is slightly a bug in java.net.NetworkInterface. -Alan. From Thomas.Salter at unisys.com Wed Dec 16 13:07:33 2009 From: Thomas.Salter at unisys.com (Salter, Thomas A) Date: Wed, 16 Dec 2009 15:07:33 -0600 Subject: nio channels with IPv6 on Windows XP In-Reply-To: <4B25FBA9.6060404@sun.com> References: <4B25FBA9.6060404@sun.com> Message-ID: I've probably strayed way off topic for this mailing list by now, but will DualStack IPv6 implementation be back-ported to Java 6 or will it always run with two stacks on Vista and Windows 7? -----Original Message----- From: Alan.Bateman at Sun.COM [mailto:Alan.Bateman at Sun.COM] Sent: Monday, December 14, 2009 3:48 AM To: Salter, Thomas A Cc: nio-dev at openjdk.java.net Subject: Re: nio channels with IPv6 on Windows XP Salter, Thomas A wrote: > It seems that AsynchronousServerSocketChannel does not work with IPv6 > sockets on Windows XP (the combination that invokes the TwoStack > implementations). Is this a known restriction or is it intended to > work? I can certainly understand why you might not want to support it. > > I tested on b77. My test runs on Windows 7 but fails on Windows XP. > It fails on the bind because an IPv6 address is passed to AF_INET socket. > As Chris said, you need Vista or newer because that's the first release of Windows with the new stack that supports dual-stack sockets (ie: interoperability with IPv4 and support for IPv4-mapped IPv6 addresses). The IPv6 stack that came in one of the Windows XP service packs was a separate stack and lacks IPv4 interoperability. -Alan. From Alan.Bateman at Sun.COM Wed Dec 16 13:20:35 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 16 Dec 2009 21:20:35 +0000 Subject: nio channels with IPv6 on Windows XP In-Reply-To: References: <4B25FBA9.6060404@sun.com> Message-ID: <4B294F23.10208@sun.com> Salter, Thomas A wrote: > I've probably strayed way off topic for this mailing list by now, but will DualStack IPv6 implementation be back-ported to Java 6 or will it always run with two stacks on Vista and Windows 7? > I'm not aware of any plans to back-port this to Sun's jdk6. So if you run on Windows Vista or 7 then NIO will always use IPv4 sockets, even if IPv6 is enabled. With jdk7 then it will create uses dual-stack IPv6 sockets if IPv6 is enabled. -Alan. From assembling.signals at yandex.ru Thu Dec 17 11:06:22 2009 From: assembling.signals at yandex.ru (assembling signals) Date: Thu, 17 Dec 2009 22:06:22 +0300 Subject: Fwd: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent, or at least javadoc clarification. Message-ID: <7351261076782@webmail13.yandex.ru> As suggested by Alan Bateman, I'm sending this mail to the nio-dev mailinglist: Hello, everyone! It seems to be complicated/confusing to rename (not move) a file (a Path) using NIO2's Path.moveTo(...). Following code is necessary: Path dir = oldFile.getParent(); Path fn = oldFile.getFileSystem().getPath(newNameString); Path target = (dir == null) ? fn : dir.resolve(fn); oldFile.moveTo(target); Got the problem and the code from here: http://stackoverflow.com/questions/1914474/how-do-i-rename-not-move-a-file-in-jdk7 Wouldn't it be good, to create a Path.renameTo(String newName) method? Or at least add clarification in javadoc? (Which is far less appropriate, an API has to "talk" to the user by classes and methods names) I'm afraid, most people will start using File.renameTo(File newFile), which would fail to handle symbolic-link correctly. Or, what is even worse, people will try to concat some strings, and create a destination Path of the result. Something like: Path newName = Paths.get(name.getParent().toString()+File.pathSeparator+"newName"); While someone would even do: name.getParent().toString()+"\\"+"newName", or name.getParent().toString()+"/"+"newName" What do you think about the problem? Best regards, Ivan G Shevchenko -- assembling dot signals at yandex dot ru From Thomas.Salter at unisys.com Thu Dec 17 11:50:20 2009 From: Thomas.Salter at unisys.com (Salter, Thomas A) Date: Thu, 17 Dec 2009 13:50:20 -0600 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or at least javadoc clarification. In-Reply-To: <7351261076782@webmail13.yandex.ru> References: <7351261076782@webmail13.yandex.ru> Message-ID: It seems like Path could use a getSibling(String name) method which effectively replaces the last node of the path with the name parameter. Then your code could be: oldFile.moveTo( oldFile.getSibling( newNameString ) ); I'd guess it's a fairly common occurrence to want to get the name of a file in the same directory as a known file, like a .h that goes with a .c file. Likewise, a variant of subpath that returns just the last node would be useful, rather than writing: int lastNode = file.getNameCount() - 1; Path last = file.subpath(lastNode, lastNode); -----Original Message----- From: nio-dev-bounces at openjdk.java.net [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of assembling signals Sent: Thursday, December 17, 2009 2:06 PM To: nio-dev at openjdk.java.net Subject: Fwd: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or at least javadoc clarification. As suggested by Alan Bateman, I'm sending this mail to the nio-dev mailinglist: Hello, everyone! It seems to be complicated/confusing to rename (not move) a file (a Path) using NIO2's Path.moveTo(...). Following code is necessary: Path dir = oldFile.getParent(); Path fn = oldFile.getFileSystem().getPath(newNameString); Path target = (dir == null) ? fn : dir.resolve(fn); oldFile.moveTo(target); Got the problem and the code from here: http://stackoverflow.com/questions/1914474/how-do-i-rename-not-move-a-file-in-jdk7 Wouldn't it be good, to create a Path.renameTo(String newName) method? Or at least add clarification in javadoc? (Which is far less appropriate, an API has to "talk" to the user by classes and methods names) I'm afraid, most people will start using File.renameTo(File newFile), which would fail to handle symbolic-link correctly. Or, what is even worse, people will try to concat some strings, and create a destination Path of the result. Something like: Path newName = Paths.get(name.getParent().toString()+File.pathSeparator+"newName"); While someone would even do: name.getParent().toString()+"\\"+"newName", or name.getParent().toString()+"/"+"newName" What do you think about the problem? Best regards, Ivan G Shevchenko -- assembling dot signals at yandex dot ru From Alan.Bateman at Sun.COM Thu Dec 17 13:19:54 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 17 Dec 2009 21:19:54 +0000 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or at least javadoc clarification. In-Reply-To: References: <7351261076782@webmail13.yandex.ru> Message-ID: <4B2AA07A.2060808@sun.com> Salter, Thomas A wrote: > It seems like Path could use a getSibling(String name) method which effectively replaces the last node of the path with the name parameter. Then your code could be: > oldFile.moveTo( oldFile.getSibling( newNameString ) ); > > I'd guess it's a fairly common occurrence to want to get the name of a file in the same directory as a known file, like a .h that goes with a .c file. > There is merit in this. Most of the time it's just foo.getParent().resolve(bar) but we have the corner-case where the path is a simple file name. The corner case means we have to something like this: Path resolveSibling(Path file, String name) { Path dir = file.getParent(); return (dir != null) ? dir.resolve(name) : file.getFileSystem().getPath(name); } > Likewise, a variant of subpath that returns just the last node would be useful, rather than writing: > int lastNode = file.getNameCount() - 1; > Path last = file.subpath(lastNode, lastNode); > Did you mean subpath(lastNode, lastNode+1)? That would return the filename and getName() already does this (just mentioning in case you missed it or I mis-understood the intention). > : > > Wouldn't it be good, to create a Path.renameTo(String newName) method? > It's also possible to rename to other locations in the file system. That's what File.renameTo(File) does. So f.renameTo(new File("bar")) will attempt to rename f into the current directory (not necessarily f's directory). Path.moveTo(Path,CopyOption...) is just more general and gives you control on if an existing file is replaced or not. There was a question on this mailing list back in September asking for a way to rename, guaranteeing that the file isn't copied. If we have that then rename becomes: source.moveTo(target, NOCOPY_ALLOWED) > Or at least add clarification in javadoc? (Which is far less appropriate, an API has to "talk" to the user by classes and methods names) > > I'm afraid, most people will start using File.renameTo(File newFile), which would fail to handle symbolic-link correctly. > Or, what is even worse, people will try to concat some strings, and create a destination Path of the result. > > Something like: > Path newName = Paths.get(name.getParent().toString()+File.pathSeparator+"newName"); > > While someone would even do: > name.getParent().toString()+"\\"+"newName", or name.getParent().toString()+"/"+"newName" > The resolve method is the easiest way to combine or join paths, so the above can be replaced with: name.getParent().resolve("newName"); -Alan. From Thomas.Salter at unisys.com Thu Dec 17 14:39:52 2009 From: Thomas.Salter at unisys.com (Salter, Thomas A) Date: Thu, 17 Dec 2009 16:39:52 -0600 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or atleast javadoc clarification. In-Reply-To: <4B2AA07A.2060808@sun.com> References: <7351261076782@webmail13.yandex.ru> <4B2AA07A.2060808@sun.com> Message-ID: As you thought, I meant getName(). I knew it was there but mixed up with the overloaded getName(index). -----Original Message----- From: nio-dev-bounces at openjdk.java.net [mailto:nio-dev-bounces at openjdk.java.net] On Behalf Of Alan Bateman Sent: Thursday, December 17, 2009 4:20 PM To: nio-dev at openjdk.java.net Subject: Re: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or atleast javadoc clarification. Salter, Thomas A wrote: > It seems like Path could use a getSibling(String name) method which effectively replaces the last node of the path with the name parameter. Then your code could be: > oldFile.moveTo( oldFile.getSibling( newNameString ) ); > > I'd guess it's a fairly common occurrence to want to get the name of a file in the same directory as a known file, like a .h that goes with a .c file. > There is merit in this. Most of the time it's just foo.getParent().resolve(bar) but we have the corner-case where the path is a simple file name. The corner case means we have to something like this: Path resolveSibling(Path file, String name) { Path dir = file.getParent(); return (dir != null) ? dir.resolve(name) : file.getFileSystem().getPath(name); } > Likewise, a variant of subpath that returns just the last node would be useful, rather than writing: > int lastNode = file.getNameCount() - 1; > Path last = file.subpath(lastNode, lastNode); > Did you mean subpath(lastNode, lastNode+1)? That would return the filename and getName() already does this (just mentioning in case you missed it or I mis-understood the intention). > : > > Wouldn't it be good, to create a Path.renameTo(String newName) method? > It's also possible to rename to other locations in the file system. That's what File.renameTo(File) does. So f.renameTo(new File("bar")) will attempt to rename f into the current directory (not necessarily f's directory). Path.moveTo(Path,CopyOption...) is just more general and gives you control on if an existing file is replaced or not. There was a question on this mailing list back in September asking for a way to rename, guaranteeing that the file isn't copied. If we have that then rename becomes: source.moveTo(target, NOCOPY_ALLOWED) > Or at least add clarification in javadoc? (Which is far less appropriate, an API has to "talk" to the user by classes and methods names) > > I'm afraid, most people will start using File.renameTo(File newFile), which would fail to handle symbolic-link correctly. > Or, what is even worse, people will try to concat some strings, and create a destination Path of the result. > > Something like: > Path newName = Paths.get(name.getParent().toString()+File.pathSeparator+"newName"); > > While someone would even do: > name.getParent().toString()+"\\"+"newName", or name.getParent().toString()+"/"+"newName" > The resolve method is the easiest way to combine or join paths, so the above can be replaced with: name.getParent().resolve("newName"); -Alan. From hjohn at xs4all.nl Sat Dec 19 16:05:40 2009 From: hjohn at xs4all.nl (John Hendrikx) Date: Sun, 20 Dec 2009 01:05:40 +0100 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or at least javadoc clarification. In-Reply-To: <4B2AA07A.2060808@sun.com> References: <7351261076782@webmail13.yandex.ru> <4B2AA07A.2060808@sun.com> Message-ID: <4B2D6A54.3050909@xs4all.nl> Alan Bateman wrote: > Salter, Thomas A wrote: >> It seems like Path could use a getSibling(String name) method which >> effectively replaces the last node of the path with the name >> parameter. Then your code could be: >> oldFile.moveTo( oldFile.getSibling( newNameString ) ); >> >> I'd guess it's a fairly common occurrence to want to get the name of >> a file in the same directory as a known file, like a .h that goes >> with a .c file. >> > There is merit in this. Most of the time it's just > foo.getParent().resolve(bar) but we have the corner-case where the > path is a simple file name. The corner case means we have to something > like this: > > Path resolveSibling(Path file, String name) { > Path dir = file.getParent(); > return (dir != null) ? dir.resolve(name) : > file.getFileSystem().getPath(name); > } I indeed do it as: path.moveTo(path.getParent().resolve(newName)); But I think in my case, path always has a valid parent. >> Wouldn't it be good, to create a Path.renameTo(String newName) method? >> > It's also possible to rename to other locations in the file system. > That's what File.renameTo(File) does. So f.renameTo(new File("bar")) > will attempt to rename f into the current directory (not necessarily > f's directory). Path.moveTo(Path,CopyOption...) is just more general > and gives you control on if an existing file is replaced or not. > > There was a question on this mailing list back in September asking for > a way to rename, guaranteeing that the file isn't copied. If we have > that then rename becomes: > source.moveTo(target, NOCOPY_ALLOWED) That might have been me. I'm still in favor of having a distinct rename method, even if it just redirects to moveTo internally, if only to avoid confusion. For my specific needs it is always clear when I want a rename or a move as the move case must be implemented by a copy+verify+delete (so I cannot let the default implementation do the move). If a move accidently occurs when I really wanted a rename it would be a bug that needed fixing. NOCOPY_ALLOWED helps to find those bugs instead of it silently doing a potentially very slow move operation. The pitfall of the current moveTo is that it can be lightning fast or potentially take hours. With copyTo I know what I'm getting in to. With a renameTo the same thing :) --John From Alan.Bateman at Sun.COM Mon Dec 21 06:14:59 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 21 Dec 2009 14:14:59 +0000 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent,or at least javadoc clarification. In-Reply-To: <4B2D6A54.3050909@xs4all.nl> References: <7351261076782@webmail13.yandex.ru> <4B2AA07A.2060808@sun.com> <4B2D6A54.3050909@xs4all.nl> Message-ID: <4B2F82E3.8070104@sun.com> John Hendrikx wrote: > That might have been me. I'm still in favor of having a distinct > rename method, even if it just redirects to moveTo internally, if only > to avoid confusion. For my specific needs it is always clear when I > want a rename or a move as the move case must be implemented by a > copy+verify+delete (so I cannot let the default implementation do the > move). If a move accidently occurs when I really wanted a rename it > would be a bug that needed fixing. NOCOPY_ALLOWED helps to find those > bugs instead of it silently doing a potentially very slow move operation. It might be right thing to do. There are a number of small API fixes, most discussed here, that I need to resolve in the new year and I'll add this to the list to examine. -Alan. PS: an alternative name for the NOCOPY_ALLOWED option is RENAME_ONLY. If combined with a few improvements to the javadoc then it would mightn't be too bad either. From uckelman at nomic.net Mon Dec 21 06:24:44 2009 From: uckelman at nomic.net (Joel Uckelman) Date: Mon, 21 Dec 2009 15:24:44 +0100 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent, or at least javadoc clarification. In-Reply-To: <4B2F82E3.8070104@sun.com> References: <7351261076782@webmail13.yandex.ru> <4B2AA07A.2060808@sun.com> <4B2D6A54.3050909@xs4all.nl> <4B2F82E3.8070104@sun.com> Message-ID: <20091221142444.4934C100D8@charybdis.ellipsis.cx> Thus spake Alan Bateman: > > PS: an alternative name for the NOCOPY_ALLOWED option is RENAME_ONLY. If > combined with a few improvements to the javadoc then it would mightn't > be too bad either. > I'm not clear on the distinction between moving and renaming here. I understand that in certian cases (e.g., when a file is moved from one filesystem to another) moving a file results in copying all of its data, while in other cases it does not (e.g., same-filesystem moves). Is this the distinction which is being made between moving (which might involve copying) and renaming (which does not involve copying)? If there could conceivably be cases where renaming could still involve a copy, then I think it's best to name the flag NOCOPY_ALLOWED to reflect what it does, rather than RENAME_ONLY, which only reflects its (presently) intended use. -- J. From Alan.Bateman at Sun.COM Tue Dec 22 06:47:52 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 22 Dec 2009 14:47:52 +0000 Subject: NIO2's Path.moveTo(...) needs renameTo(...) aequivalent, or at least javadoc clarification. In-Reply-To: <20091221142444.4934C100D8@charybdis.ellipsis.cx> References: <7351261076782@webmail13.yandex.ru> <4B2AA07A.2060808@sun.com> <4B2D6A54.3050909@xs4all.nl> <4B2F82E3.8070104@sun.com> <20091221142444.4934C100D8@charybdis.ellipsis.cx> Message-ID: <4B30DC18.3050007@sun.com> Joel Uckelman wrote: > : > I'm not clear on the distinction between moving and renaming here. I > understand that in certian cases (e.g., when a file is moved from one > filesystem to another) moving a file results in copying all of its data, > while in other cases it does not (e.g., same-filesystem moves). Is this > the distinction which is being made between moving (which might involve > copying) and renaming (which does not involve copying)? > > If there could conceivably be cases where renaming could still involve > a copy, then I think it's best to name the flag NOCOPY_ALLOWED to reflect > what it does, rather than RENAME_ONLY, which only reflects its (presently) > intended use. > > That is fair comment as we can't make assumptions for all providers. At this time, the moveTo method doesn't make any distinction, in the javadoc, between renaming and moving. That is, source.moveTo(target) or source.moveTo(target, REPLACE_EXISTING) should "just work" and it doesn't matter if you are simply giving a file a new name in the same directory or moving the file to another directory (that is on the same or a different file system/volume). In the case of the default provider, then it will always attempt to rename the file and only fall back to a copy+delete if the rename fails because the target is a different file system/volume. If the target is associated with a different provider then moveTo will always copy+delete (as rename doesn't make sense). The suggestion that has come up, is to introduce a new rename method that will be "fast" or rather won't fall-back to a copy+delete. The ATOMIC_MOVE option actually does this already (at the implementation level anyway) but that option is for a different use-case and can't guarantee behavior when the target already exists. -Alan. From jdavis at pcprogramming.com Tue Dec 29 02:32:37 2009 From: jdavis at pcprogramming.com (John Davis) Date: Tue, 29 Dec 2009 04:32:37 -0600 Subject: GetQueueCompletionStatusEx Message-ID: Any plans to add the new 2008 Server API's to nio? GetQueuedCompletionStatusEx SetFileIoOverlappedRange SetFileCompletionNotificationModes JD -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20091229/cc7b812f/attachment.html From Alan.Bateman at Sun.COM Thu Dec 31 08:40:56 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 31 Dec 2009 16:40:56 +0000 Subject: GetQueueCompletionStatusEx In-Reply-To: References: Message-ID: <4B3CD418.8000406@sun.com> John Davis wrote: > Any plans to add the new 2008 Server API's to nio? > > GetQueuedCompletionStatusEx Maybe, but it depends on the configuration. With a fixed thread pool then each thread retrieves a completion result and dispatches directly to the completion handler. For that case it would be inconvenient to retrieve completion results in batch. However for a cached or other thread pool then there are 1+ internal threads where each thread retrieves a completion result and submits a task to the thread pool to invoke the completion handler. For that configuration then it may be beneficial (esp. when there is a single internal thread retrieving events) but we would need to measure it before changing the code. Have you done any tests or do you have performance results that you could share on this one? > SetFileIoOverlappedRange No specific plans at this point because it requires special privileges to lock pages into memory. However if there was some measurable improvement for privileged users then it wouldn't be too difficult to make of use of this as there is already a cache of OVERLAPPED structures per channel (or handle). > SetFileCompletionNotificationModes This one is new (to me anyway). In the implementation we don't use a completion events. If Windows is still setting some internal event and this can be skipped then it could be beneficial. We require a completion result per successful I/O operation so we couldn't use that mode. -Alan. From jdavis at pcprogramming.com Thu Dec 31 09:03:54 2009 From: jdavis at pcprogramming.com (John Davis) Date: Thu, 31 Dec 2009 11:03:54 -0600 Subject: GetQueueCompletionStatusEx In-Reply-To: References: <4B3CD418.8000406@sun.com> Message-ID: Forgot the CC On Thu, Dec 31, 2009 at 11:01 AM, John Davis wrote: > I'm not an expert on the nio internals. My thinking is it would be nice if > everything could be done with a single thread (ie all reads/writes on all > handles are async and pumped thru IOCP). Because the kernel context switch > overhead is pretty much eliminated with this call, there is no longer a need > to have a thread pool to handle the context switch overhead. We also > wouldn't have the batching dilema with the thread pool gone. Only problem > with this model is that there's no way to guarantee a completion handler > won't block. > > My experience from work is it's fast as hell in straight C++. Due to > NDA's, etc. I don't have any benchmarks to show. > SetFileCompletionNotificationModes, allows the callback to be invoked > immediately, rather than pumping thru IOCP and thread pool. > > On Thu, Dec 31, 2009 at 10:40 AM, Alan Bateman wrote: > >> John Davis wrote: >> >>> Any plans to add the new 2008 Server API's to nio? >>> GetQueuedCompletionStatusEx >>> >> Maybe, but it depends on the configuration. With a fixed thread pool then >> each thread retrieves a completion result and dispatches directly to the >> completion handler. For that case it would be inconvenient to retrieve >> completion results in batch. However for a cached or other thread pool then >> there are 1+ internal threads where each thread retrieves a completion >> result and submits a task to the thread pool to invoke the completion >> handler. For that configuration then it may be beneficial (esp. when there >> is a single internal thread retrieving events) but we would need to measure >> it before changing the code. Have you done any tests or do you have >> performance results that you could share on this one? >> >> SetFileIoOverlappedRange >>> >> No specific plans at this point because it requires special privileges to >> lock pages into memory. However if there was some measurable improvement for >> privileged users then it wouldn't be too difficult to make of use of this as >> there is already a cache of OVERLAPPED structures per channel (or handle). >> >> SetFileCompletionNotificationModes >>> >> This one is new (to me anyway). In the implementation we don't use a >> completion events. If Windows is still setting some internal event and this >> can be skipped then it could be beneficial. We require a completion result >> per successful I/O operation so we couldn't use that mode. >> >> -Alan. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20091231/adcd3a3b/attachment.html From jdavis at pcprogramming.com Thu Dec 31 09:23:11 2009 From: jdavis at pcprogramming.com (John Davis) Date: Thu, 31 Dec 2009 11:23:11 -0600 Subject: Fwd: GetQueueCompletionStatusEx In-Reply-To: References: <4B3CD418.8000406@sun.com> Message-ID: Alzheimers must be setting in, forgot the CC again ... ---------- Forwarded message ---------- From: John Davis Date: Thu, Dec 31, 2009 at 11:22 AM Subject: Re: GetQueueCompletionStatusEx To: Alan Bateman This is also noteworthy ... While you don?t need an event in the Overlapped structure when using Completion Ports because the event is never waited on, if you leave it out, the event in the file handle will be set and that incurs extra locking. http://blogs.technet.com/winserverperformance/archive/2008/06/26/designing-applications-for-high-performance-part-iii.aspx On Thu, Dec 31, 2009 at 11:01 AM, John Davis wrote: > I'm not an expert on the nio internals. My thinking is it would be nice if > everything could be done with a single thread (ie all reads/writes on all > handles are async and pumped thru IOCP). Because the kernel context switch > overhead is pretty much eliminated with this call, there is no longer a need > to have a thread pool to handle the context switch overhead. We also > wouldn't have the batching dilema with the thread pool gone. Only problem > with this model is that there's no way to guarantee a completion handler > won't block. > > My experience from work is it's fast as hell in straight C++. Due to > NDA's, etc. I don't have any benchmarks to show. > SetFileCompletionNotificationModes, allows the callback to be invoked > immediately, rather than pumping thru IOCP and thread pool. > > On Thu, Dec 31, 2009 at 10:40 AM, Alan Bateman wrote: > >> John Davis wrote: >> >>> Any plans to add the new 2008 Server API's to nio? >>> GetQueuedCompletionStatusEx >>> >> Maybe, but it depends on the configuration. With a fixed thread pool then >> each thread retrieves a completion result and dispatches directly to the >> completion handler. For that case it would be inconvenient to retrieve >> completion results in batch. However for a cached or other thread pool then >> there are 1+ internal threads where each thread retrieves a completion >> result and submits a task to the thread pool to invoke the completion >> handler. For that configuration then it may be beneficial (esp. when there >> is a single internal thread retrieving events) but we would need to measure >> it before changing the code. Have you done any tests or do you have >> performance results that you could share on this one? >> >> SetFileIoOverlappedRange >>> >> No specific plans at this point because it requires special privileges to >> lock pages into memory. However if there was some measurable improvement for >> privileged users then it wouldn't be too difficult to make of use of this as >> there is already a cache of OVERLAPPED structures per channel (or handle). >> >> SetFileCompletionNotificationModes >>> >> This one is new (to me anyway). In the implementation we don't use a >> completion events. If Windows is still setting some internal event and this >> can be skipped then it could be beneficial. We require a completion result >> per successful I/O operation so we couldn't use that mode. >> >> -Alan. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20091231/a28d294a/attachment.html