Race Condition in result set wrapper because of the ColumnNameCache
Description
Environment
Activity
Emanuel Wlaschitz February 20, 2014 at 9:29 AMEdited
-This also seems to cause (and fix) an issue with the Oracle data client. Querying an XML Column (using the usertype XDocType) seemed to return a wrong column index or something, where the Oracle Client called OracleDataReader.GetDecimal on a XML column - which obviously fails.- Turns out it doesn't, was a random hit where it actually worked indeed.
Disabling the wrapper fixes it , and the pull request seems to do the trick aswell.
Oskar Berggren October 2, 2013 at 8:54 PM
Pull request merged to master in 938702c904ac22b828b91668373a965fdba5c5ac.
GrzP August 8, 2013 at 8:06 AM
I added pull request with fix proposition: https://github.com/nhibernate/nhibernate-core/pull/219.
Jozsef Kanczler August 6, 2013 at 2:05 PM
Yeah, that was the issue for us. The internal code of the dictionary throws an index was outside the bounds of the array when something goes wrong because concurrency. We implemented a workaround. We implemented a custom data reader that can cache column names. It still improved our performance for Sybase. Unfortunately you have to wrap database commands, data readers and have to implement a custom nhibernate driver. And of course we turned off this NHibernate feature.
GrzP August 6, 2013 at 12:35 PM
I have adonet.wrap_result_sets set to true. (Unfortunately I'm running NHibernate 3.1.0.40000, because of an error in the newer version).
I'm running multiple threads with separate ISession created per thread, just reading data. Ocassionally (very seldom) I get an error when accessing lazy loaded collection. The error seems to to have the same origin: AdoNet.ResultSetWrapper/ ColumnNameCache.GetIndexForColumnName:
Do you think this is the same issue? :
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at NHibernate.AdoNet.ColumnNameCache.GetIndexForColumnName(String columnName, ResultSetWrapper rs)
at NHibernate.AdoNet.ResultSetWrapper.GetOrdinal(String name)
at NHibernate.Type.NullableType.NullSafeGet(IDataReader rs, String name)
at NHibernate.Type.NullableType.NullSafeGet(IDataReader rs, String[] names, ISessionImplementor session, Object owner)
at NHibernate.Type.AbstractType.Hydrate(IDataReader rs, String[] names, ISessionImplementor session, Object owner)
at NHibernate.Type.ComponentType.Hydrate(IDataReader rs, String[] names, ISessionImplementor session, Object owner)
at NHibernate.Type.ComponentType.NullSafeGet(IDataReader rs, String[] names, ISessionImplementor session, Object owner)
at NHibernate.Persister.Collection.AbstractCollectionPersister.ReadKey(IDataReader dr, String[] aliases, ISessionImplementor session)
at NHibernate.Loader.Loader.ReadCollectionElement(Object optionalOwner, Object optionalKey, ICollectionPersister persister, ICollectionAliases descriptor, IDataReader rs, ISessionImplementor session)
at NHibernate.Loader.Loader.ReadCollectionElements(Object[] row, IDataReader resultSet, ISessionImplementor session)
at NHibernate.Loader.Loader.GetRowFromResultSet(IDataReader resultSet, ISessionImplementor session, QueryParameters queryParameters, LockMode[] lockModeArray, EntityKey optionalObjectKey, IList hydratedObjects, EntityKey[] keys, Boolean returnProxies)
at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.LoadCollection(ISessionImplementor session, Object id, IType type)
Details
Assignee
Oskar BerggrenOskar BerggrenReporter
Jozsef KanczlerJozsef KanczlerComponents
Fix versions
Affects versions
Priority
MajorWho's Looking?
Open Who's Looking?
Details
Details
Assignee
Reporter
Components
Fix versions
Affects versions
Priority
Major
In some cases the NHibernate stores 'static' instance of Loader class. The loader class has a column name cache if result set wrapping is turned on in the configuration. The column name cache itself has a dictionary that is not thread safe, because of this different threads have access to the same dictionary. The dictionary implementation can be ended in an infinite loop if it's not handled thread safely.
We're using Sybase database in a load test with result set wrapping (performance consideration). Sometimes our IIS process starting to use 100% CPU. In a memory dump we see that lots of threads is in the Dictionary FindEntry method, that is called from the ColumnNameCache.GetIndexForColumnName.
Unfortunately we can't create a test case for NHibernate, because it's a race condition, but we could verify that something goes wrong with debugging:
using (var session = this.OpenSession())
{
session.Get<Person>(person.Id);
}
using (var session = this.OpenSession())
{
session.Get<Person>(person.Id);
}
We put a breakpoint in the column name cache. After calling session.Get<Person>(person.Id) method, the dictionary of the column name cache already contained an element. That means after creating a new session, we still had the same dictionary.