Data Mapping Issue.. sometimes..
Data Mapping Issue.. sometimes..
I've had this come up twice, once while debugging and another time as a bug report submission from a Dual Xeon (NUMA) system.
I've never been able to reproduce it until yesterday when I tried FastMM5 and it showed up.
Once I also turned on MadExcept Leak reporting it was a breeze to reproduce.
I traced it to TCRRecordSet.GetFieldData in particular Line: 4376/4377.
On rare occasions accessing TCRFieldDesc(Field).MapRule gives an Access Violation.
Commenting those two lines out made everything work fine since I only had two Data Mappings and neither used them.
An attempt to use Breakpoints to log data using Codesite eliminated the error.
After evaluating the Mapping code I decided that using MACROS and CAST in the two SQL statements was easier.
But MapRule is never nil yet at some point it must have been freed or is pointing nowhere.. not going to attempt to try and duplicate this but you should know..
I've never been able to reproduce it until yesterday when I tried FastMM5 and it showed up.
Once I also turned on MadExcept Leak reporting it was a breeze to reproduce.
I traced it to TCRRecordSet.GetFieldData in particular Line: 4376/4377.
On rare occasions accessing TCRFieldDesc(Field).MapRule gives an Access Violation.
Commenting those two lines out made everything work fine since I only had two Data Mappings and neither used them.
An attempt to use Breakpoints to log data using Codesite eliminated the error.
After evaluating the Mapping code I decided that using MACROS and CAST in the two SQL statements was easier.
But MapRule is never nil yet at some point it must have been freed or is pointing nowhere.. not going to attempt to try and duplicate this but you should know..
Re: Data Mapping Issue.. sometimes..
Thank you for bringing this to our attention. Could you send us a sample project demonstrating the issue?
Re: Data Mapping Issue.. sometimes..
As stated I've simply removed 'Data Type Mapping' because I was not able to reproduce until I added FastMM5 and Leak Checking..
In short I have no incentive to go and see what could possibly be the issue. But threading would be in my list of things to consider. Here is why:
In short I have no incentive to go and see what could possibly be the issue. But threading would be in my list of things to consider. Here is why:
- Reproducing this requires me to run several threaded Data Modules with db access first, yet none use 'Data Type Mapping'
- By the time that Mapping code in GetFieldData is called all those threaded connections are terminated and back in Pooling
- Yet simply adding a Breakpoint with CodeSite logging stops the exception, CodeSite uses a Critical section
- That said all calling code is guaranteed to be execute in the Main thread because an assertion already checks that.
Re: Data Mapping Issue.. sometimes..
Each time the Dataset is opened it creates a new MapRule entry for the Collection, but that is not updated.
All runs until the first memory is reallocated, mostly the string (Format) because its a managed type.
Data Type Map and Field Definition:
Callstack:
Code Modifications and Breakpoints **:
CodeSite:
Breakpoints: https://imgur.com/a/EhVx7g5
All runs until the first memory is reallocated, mostly the string (Format) because its a managed type.
Data Type Map and Field Definition:
Code: Select all
DataTypeMap = <
item
FieldName = 'FETCHHASSNAPSHOT'
FieldType = ftBoolean
end>
object LookupDomainsFETCHHASSNAPSHOT: TBooleanField
FieldName = 'FETCHHASSNAPSHOT'
ReadOnly = True
end
Code: Select all
main thread ($12ac):
0048b21a +0eda MyApp.exe System.SysUtils WideFormatBuf
00489d20 +0020 MyApp.exe System.SysUtils FormatBuf
00489f79 +0099 MyApp.exe System.SysUtils FmtStr
00489ea5 +0015 MyApp.exe System.SysUtils Format
00489e77 +0017 MyApp.exe System.SysUtils Format
028fba84 +0264 MyApp.exe CRAccess 4378 +16 TCRRecordSet.GetFieldData
0279e93e +017e MyApp.exe MemData 3222 +16 TData.GetField
Code: Select all
procedure TCRRecordSet.GetFieldData(Field: TFieldDesc; FieldBuf: IntPtr; Dest: IntPtr; NeedConvert: boolean);
{$IFNDEF LITE}
var
Buf: IntPtr;
Status: TConvertStatus;
Value: Variant;
Str: string;
s : string; fd : TCRFieldDesc absolute Field;
{$ENDIF}
begin
{$IFNDEF LITE}
if NeedConvert and (TCRFieldDesc(Field).OnDemandConverter <> nil) then begin
Buf := GetBufForDataMappingConverter(FieldBuf, Field.DataType, Field.HasParent, True);
s := Format('Type: %d, Addr: %x', [Ord(Field.DataType), UIntPtr(fd.MapRule)]);
s := Format('>> MapRule.IgnoreErrors: %s', [BoolToStr(fd.MapRule.IgnoreErrors, True)]); **
try
FConvertInfo.StringHeap := FStringHeap; **
FConvertInfo.Source := Buf;
FConvertInfo.SourceOffset := 0;
FConvertInfo.SourceLen := Field.Length;
FConvertInfo.SourceScale := Field.Scale;
FConvertInfo.Dest := Dest;
FConvertInfo.DestOffset := 0;
FConvertInfo.DestLen := TCRFieldDesc(Field).MapLength;
FConvertInfo.DestScale := TCRFieldDesc(Field).MapScale;
s := Format('>> Try MapRule.Format: %s', [fd.MapRule.Format]);
FConvertInfo.IgnoreConvertErrors := TCRFieldDesc(Field).MapRule.IgnoreErrors; **
FConvertInfo.Format := TCRFieldDesc(Field).MapRule.Format;
...
Code: Select all
function TCRMapRules.AddRule(const FieldName: string;
DBType: Word; DBLengthMin, DBLengthMax, DBScaleMin, DBScaleMax: Integer;
DataType: Word; FieldLength, FieldScale: Integer;
IgnoreErrors: boolean; const Format: string): TCRMapRule;
var
Error: Exception; s : string;
begin
Error := DBTypeInfos.Check(DBType, DBLengthMin, DBLengthMax, DBScaleMin, DBScaleMax, FieldName = '');
Result := nil;
if Error = nil then begin
Result := GetMapRuleClass.Create(Self);
s := Sysutils.Format('AddMapRule.Index: %d - Addr: %x', [Result.Index, UIntPtr(Result)]);
Result.FieldName := FieldName; **
...
Code: Select all
AddMapRule.Index: 0 - Addr: 1D421DC8
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format:
AddMapRule.Index: 0 - Addr: 19795718
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format:
AddMapRule.Index: 0 - Addr: 19795718
Type: 9, Addr: 1D421DC8
>> MapRule.IgnoreErrors: False
>> Try MapRule.Format: Kaput**
Re: Data Mapping Issue.. sometimes..
We haven't been able to reproduce the issue. If you'd like us to investigate and fix it, please send us a complete sample project demonstrating the issue.
Re: Data Mapping Issue.. sometimes..
Ok, we'll try to investigate further. Thank you for your help.