I believe we can get same results by “Regular Expression”， but it’s a little complex. The above function will be more simple and faster.

On the other hand, some Emeditor uses have no experiences on “Regular Expression”, if possible to simplify the filter function, such as use “Wildcard” as keywords to fuzzy filter.
For example, P?T to match pit, pot, put………; P*t to match pt, put,pot, part,past…….This could be default behavior. If more complex situation, user can use “Regular Expression”.

This option could typically be implemented as a second (optional) flags parameter to the filters.Import() method, which would make it fully backwards compatible too.

As you can understand, this would make it possible for macros to perform extremely complex dynamic filtering of documents, depending on the logic of the macro itself, while the filter templates that it uses can still be originally created by humans and stored in the different filter files that are imported by the macro, step-by-step, in this way.

Pretty please with sugar on top could we have this new flag for document.filters.Import()? :-)

Here is a proof of concept for reproduction of the bug:
——————————————————————————

Input CSV file (only containing one column, but the bug behavior does not appear if loaded as normal text file, so it must be loaded as CSV):

something here
TEST\something.exe
TEST\something.exe

Source code of the macro that is executed on it (with a shortcut key, while CSV mode is active, and either the cell cursor is only placed on the entire middle “TEST\something.exe” cell, OR any arbitrary substring is selected inside this cell):

if(document.CellMode)
{
	document.Filter(document.selection.Text, document.selection.GetActivePointX(eePosCell), eeFindKeepPrevious | eeFindNegative);
}
else
{
	document.Filter(document.selection.Text, 0, eeFindKeepPrevious | eeFindNegative);
}

The expected result is that only line 1 should remain visible, but what happens is that ALL lines remain visible.

My guess is that the following has something to do with:

1.
The “Use Escape Sequence” option is automatically used in the created filter now (but not before?), which then chokes on the “\s” embedded in this string.

2.
No matter if I select any substring inside the cell before executing the macro with my shortcut key for it, the entire text of the cell is still reported in “document.selection.Text” in the macro code, which has not been the case previuosly, which I thus assume is a bug too?

Could you possibly please take a look at this as soon as possible, since it is currently breaking my entire workflow and use case for EmEditor?

Here is a proof of concept for reproduction of the bug:
——————————————————————————

Input CSV file:

1,/cgi-bin/listserv/wa.exe?SHOWTPL=<script>alert(/openvas-xss-test/)</script>
2,abcd
3,/scripts/wa.exe?SHOWTPL=<script>alert(/openvas-xss-test/)</script>

Source code of the macro that is executed on it (while CSV mode is active, and the string “openvas” is selected in the editor window):

if(document.CellMode)
{
	document.Filter(document.selection.Text, document.selection.GetActivePointX(eePosCell), eeFindKeepPrevious);
}
else
{
	document.Filter(document.selection.Text, 0, eeFindKeepPrevious);
}

The expected result is that only line 2 should be filtered away, leaving line 1 and 3 visible, BUT instead line 3 is also removed, leaving only line 1 visible!

NOTE: This bug only seems to happen in Cell mode (CSV in my case), but NOT in normal document mode even if using the exact same document contents, which may be an important fact for your bug reproduction/debugging!

What makes me even more scared is that if I remove the “<script>” and “</script>” parts from the cell contents, it works as expected again, so this might even be some kind of unintended script execution vulnerability/bug in EmEditor or similar (similar to cross-site scripting on web pages)!

Could you please take a look at this as soon as possible, since it is breaking my entire use case for EmEditor, and may even be some kind of security vulnerability!?

	document.selection.SetActivePoint(eePosCell, document.selection.GetActivePointX(eePosCell), 2, false);
	document.selection.SetActivePoint(eePosCell, document.selection.GetActivePointX(eePosCell), document.GetLines() - 1, true);

It works great when there is no active filter, but as soon as any line at the end of the file is currently filtered-out, it won’t work. It will then instead select all the columns including and to the left of the column where the “cell cursor” is currently at.

The simple explanation for this is that the expression “document.GetLines() – 1” will not move the cursor from the last (empty) line above it in the filtered data (because the line number of that line will still be less than “document.GetLines() – 1” if any lines have been filtered-out inbetween the end of the document and the last visible (= non-filtered out) line in the document). Thus, the cell cursor will “snap left” on the very last line of the file (which is always empty, therefore bringing the cell cursor all the way to the left, consequently selecting also all the columns to the left of the intended column).

So, my question is therefore: Is there any way to get the line numbers of only the visible lines (i.e. those that are not hidden by the filter) in a file, from a macro?

If not, how can I go about selecting the entire contents of a single column from a macro in the above described scenario?

Example data (with example line numbers to the left):

1: col1;col2;col3
2: 1;2;3
3: 11;22;33
4: 111;222;333
10:

In this case, lines 5-9 of the file have been filtered-out by a filter expression, therefore not being visible in the editor window.

So, what I want in this example is to get the number of the last visible line (i.e. nr 4, which is the last visible line before the very last line which in this case is nr 10, where the last one is always empty without any cells, except the “artificial” cell being placed to the very left), or any other method to reliably select the entire data contents (i.e. rows 2-4) of this column.

Are any of these two things currently possible from EmEditor macros, or could you otherwise possibly add the possibility to do any of these two things?

Doing this, I have noticed a workflow-disrupting detail that can best be described as a bug, or at least “unintended undesirable behavior” in connection with filtering out rows.

To be more specific, the cell (column number) having focus after having filtered away a certain row depends on the lengths of the values inside the cells of the row that was just filtered away. Or, put in another way: After having filtered away a row, a cell in a “random” column will be focused/selected (i.e. getting the mint-green “cell cursor”), instead of focus staying inside the same column as previous.

I have put together a simple PoC for you to better be able to analyze this problem:

1.
Create a CSV document containing only the following two rows, and open this document in CSV mode (“Comma separated”, Cell Selection Mode):

1,filter_this,3
1,2,3

2.
Add a negative filter for the value “filter_this” in column 2, thus removing the first line from view.

For extra clarity, here is what this filter looks like when exported to a tsv file:
on filter_this 1 N 0 -1

Also, if it matters, I’m performing this filtering by invoking the following macro (using a keyboard shortcut) while having the “filter_this” cell active:
document.Filter(document.selection.Text, document.selection.GetActivePointX(eePosCell), eeFindKeepPrevious | eeFindNegative);

3.
As you will see, the first row will now be filtered away (just as expected), and the active cell (in the remaining row 2) will still be column 2, i.e. the same column that was active when we invoked the filter (just as expected).

4.
Now, do it all again, but with the only difference that the value of the first cell on the first line is first changed from “1” to “123”. That is, like this:

123,filter_this,3
1,2,3

The active cell after the filtering (i.e. step 3 above) will now consistently be in column 3 instead (i.e. NOT the same column that was active before the filtering), which is very workflow-breaking (i.e. requiring the user to localize which random column he has just been moved too before being able to continue work with e.g. filtering out more values or similar).

So, it would be really great if this behavior could be changed to be consistent (staying in the same cell column) instead of “random” like now, could you possibly please fix this?

(my guess would be that the previous “character column number” is currently used instead of the “cell column number” to decide the new active cell after the filtering, so in that case simply make sure to use the “cell column number” for this instead as long as you’re in Cell Selection Mode when the filtering occurs)

The following regular expression works as expected (finds and colors green all lines that do not begin with a tab):

^[^\t].*

As soon as I add a linebreak though, the green-coloring of search results stops working completely. The actual finding of the lines still works though (the lines are selected in grey and jumped to correctly when pressing “Find next”):

^[^\t].*\n

If adding another linebreak to the regular expression though, both the green-coloring and the correct finding of the lines stop working completely though, and seemingly completely random lines are identified as “found” (I tried bookmarking the found lines from the Find dialog, since this was the only way of identifying them at all with the highlighting not working correctly):

^[^\t].*\n\n

I am not using any special search settings, and all checkboxes are unchecked in the “Advanced…” dialog of the Find dialog. Also I tried using both regular expression engines without any difference.

ADDITION:
I just noticed that the inability to find the correct lines happens only when the filter function is used (typically when there is really a line following the seemingly single lines, but it has been filtered away), but the lack-of-green-coloring bug happens no matter if there is a filter applied or not, so it’s a separate bug.

Here is a test case:

Try to use the regular expression ^[^\t].*\n\n on the following, without any filtering, and you will see that the green-coloring of search results will be missing:

aaa

bbb

ccc
111

ddd

eee

Now, filter away the line containing the “111” (and a tab before it) and try to search again with the same regexp, and you will see that the “ccc” line will still not be matched, although it should be now (since it is also a “single line” after that filtering-away of the subsequent “111” line).

I’m using the following macro to filter out lines from my documents:

document.Filter(document.selection.Text, 0, eeFindKeepPrevious | eeFindNegative);

If it would be of relevance, I have this macro bound to the shortcut ctrl+d, which is what I’m using to execute it.

The following example line in a text document can be used as a bug PoC:

abc\def\ghi

If I select the part “ghi” from the line above and execute my macro above (by pressing its shortcut, as described above), the line is filtered-away, just as expected.

But, if I (after bringing it back again from the previous test case, of course) select the part “def\ghi” from the line and execute my macro above, it does NOT disappear as expected. It simply stays visible. If I look in the advanced filter dialog box, a filter entry has been added though, as follows, but seemingly still doesn’t work for some reason:

def\\ghi | Whole line | EN

And finally, even stranger, if I select the entire line (i.e. “abc\def\ghi”) and execute the macro, the following happens:

1. The line still does not disappear.
2. The line turns orange (as if there was a marker defined for it, but there is no marker defined with that color in my editor).
3. (even stranger) The line is marked as “edited” in the editor (i.e. a yellow indicator to the left of the line), which I really don’t understand why, since my macro only performs a filter operation!?
4. An entry is still created in the filter (abd\\def\\ghi | Whole line | EN), but apparently still not working.

One guess would be that these backslash related problems were introduced as a result of the fix you implemented for my previous bug report regarding filter lines containing TAB characters (https://www.emeditor.com/forums/topic/exported-filters-containing-tab-cannot-be-re-imported/)?

Scripting the import (and export) operation itself (with a possibility to provide the filename to be imported/exported) would be highly useful, so my question is if this is already possible, or if it could otherwise be implemented in the macro capabilities of EmEditor?